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EXECUTIVE  SUMMARY 


This  report  describes  the  evaluation  of  the  Aviation  Weather 
Research  Products  (AWRP)  conducted  by  ACT-320  at  the  Federal 
Aviation  Administration  (FAA)  Technical  Center  from  July  17  to 
August  25,  1995.  Specific  results,  conclusions,  and 
recommendations  for  the  evaluation  are  detailed  within  the 
report. 

The  AWRP  software  produces  graphical  aviation' weather  products 
specifically  intended  for  use  by  nonmeteorologists  such  as  Air 
Route  Traffic  Control  Center  (ARTCC)  Traffic  Management  Unit 
(TMU)  Coordinators  and  Automated  Flight  Service  Station  (AFSS) 
Specialists.  The  products  are  created  using  National  Weather 
Service  (NWS)  computer  forecasted  gridded  data,  radar  data,  and 
surface  observations.  A  precursor  to  the  AWRP,  the  Aviation 
Weather  Products  Generator  (AWPG) ,  provided  specific  product 
recommendations  and  user  requests  to  the  format  and  displdy  of 
the  AWRP. 

While  the  AWRP  is  not  a  formal  FAA  acquisition,  the  products  are 
designed  to  be  coupled  with  existing  and  future  weather  systems. 
The  products  are  expected  to  significantly  improve  the  accuracy 
and  timeliness  of  current  and  forecasted  aviation  weather 
information.  Thus,  the  purpose  of  the  evaluation  was  to  identify 
AWRP  concepts  and  techniques  that  demonstrate  utility  and  which 
can  be  incorporated  into  FAA  acquisition  programs  and  commercial 
vendor  supplied  weather  systems ^ 

The  AWRP  evaluation  was  conducted  from  a  Human  Factors, 
Meteorological,  and  System  Administration  perspective.  The  Human 
Factors  portion  of  the  evaluation  assessed  AWRP  user- system 
interface  and  the  AWRP  User ' s  Guide  in  addition  to  determining 
the  extent  to  which  the  AWRP  met  the  job  task  and  weather 
information  needs  of  controllers  and  specialists.  The 
Meteorological  portion  of  the  evaluation  determined  the  amount  of 
meteorological  interpretation  needed  for  the  individual  products, 
whether  products  showed  improvement  over  existing  operational 
products,  and  meteorological  consistency  within  and  between 
products.  The  System  Administration  portion  of  the  evaluation 
addressed  the  usability  and  effectiveness  of  the  AWRP 
installation  and  system  management  documents. 
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Although  the  evaluation  identified  improvements  over  current 
systems,  several  additional  improvements  would  have  to  be  made  to 
the  AWRP  in  order  to  consider  the  products  for  potential 
incorporation  into  FAA  systems.  These  include: 

a.  improvements  to  the  AWRP  display  and  interface  according 
to  human  factors  guidelines  and  standards; 

b.  incorporation  of  all  reasonable  user  requests; 

c.  addition  of  all  required  weather  information; 

d.  resolution  of  interpretation  issues  concerning 
conflicting  information; 

e.  improvements  in  meteorological  consistency;  and 

f.  improvements  to  the  system  administration  documentation. 

Despite  the  discrepancies  noted,  the  AWRP  has  the  potential  to 
provide  controllers  and  specialists  with  much  needed  improvements 
in  the  area  of  aviation  weather  information.  ACT-320  strongly 
recommends  the  continued  development  of  aviation  weather 
graphical  products  and  the  underlying  science.  Additionally,  if 
further  development  of  the  AWRP  is  going  to  be  conducted,  it  is 
recommended  that  suggested  improvements  be  made  to  the  display, 
interface,  User's  Guide,  system  administration  documentation,  and 
meteorological  interpretation  and  consistency.  However, 
consideration  should  be  given  to  off-the-shelf  commercial  weather 
products  or  the  development  of  weather  graphics  by  commercial 
vendors . 


vii 


1.  INTRODUCTION. 

The  Federal  Aviation  Administration  (FAA)  Weather  Processors 
Integrated  Product  Team  manages  research  and  development  of 
aviation  weather  products.  Products  and  related  science  are 
developed  by  several  laboratories  such  as  the  National  Center  for 
Atmospheric  Research  (NCAR) ,  the  Forecast  Systems  Laboratory  ; 
(FSL) ,  and  Lincoln  Laboratories  among  others.  When  these  weather 
products  are  coupled  with  existing  and  future  weather  systems, 
they  are  expected  to  significantly  improve  the  accuracy  and 
timeliness  of  current  and  forecasted  weather  information 
available  to  Automated  Flight  Service  Stations  (AFSSs) ,  Air  Route 
Traffic  Control  Centers  (ARTCCs) ,  and  private  aviation  weather 
users . 

In  the  summer  of  1993,  the  Aviation  Weather  Products  Generator 
(AWPG)  underwent  an  initial  product  Demonstration  and  Evaluation 
(the  1993  Demval) .  The  1993  Demval  was  conducted  by  the  FAA 
Technical  Center  and  NCAR.  The  1993  Demval  yielded  specific 
product  recommendations  along  with  general  conclusions  and 
recommendations  to  tailor  aviation  weather  products  and  functions 
to  user  group  needs. 

In  addition  to  user  feedback  gathered  during  the  1993  Demval,  the 
Air  Traffic  User  Working  Group  (ATUWG)  and  a  Job  Task  Analysis 
(JTA)  completed  by  ACT-320  have  provided  operationally  based 
information  requirements  for  use  in  the  review  of  the  Aviation 
Weather  Research  (AWR)  graphical  weather  products.  ATUWG 
consists  of  controllers  and  specialists  who  are  representative  of 
operational  air  traffic  users.  Members  of  the  ATUWG  met  on  a 
semiannual  basis  to  review  the  progress  of  the  Aviation  Weather 
Research  Products  (AWRP)  and  provide  feedback. 

The  JTA  effort  was  conducted  by  ACT-320  to  better  understand  the 
use  of  weather  products  by  Traffic  Management  Coordinators  (TMCs) 
at  ARTCC  Traffic  Management  Units  (TMU)  and  AFSS  Specialists. 

The  JTA  was  intended  to  provide  baseline  information  regarding 
current  AFSS  and  ARTCC  TMU  job  task  and  weather  information  needs 
for  the  development  of  AWPG  graphical  weather  products.  JTA 
results  outline  how  specific  weather  information  is  obtained,  why 
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it  is  obtained,  and  the  format  of  the  weather  information.  These 
results  can  be  used  to  review  graphical  weather  products  in  order 
to  identify  how  well  the  products  satisfy  job  task  requirements. 

A  successor  to  the  AWPG,  the  AWR  graphical  weather  products 
include  various  weather  products  resulting  from  aviation  weather 
research.  These  products  are  created  using  National  Weather 
Service  (NWS)  computer  forecasted  gridded  data,  radar  data,  and 
surface  observations.  The  AWR  graphical  weather  products 
software  produces  user-specific  aviation  weather  products 
specifically  intended  for  use  by  nonmeteorologists.  Potential 
users  of -the  aviation  weather  products  include  ARTCC  TMU 
coordinators,  AFSS  Specialists,  and  airport  operations  personnel. 

While  the  AWR  graphical  weather  products  are  not  designated  as  a 
formal  FAA  acquisition,  the  concepts  and  techniques  identified  as 
having  utility  can  be  incorporated  into  FAA  acquisition  programs 
such  as  the  Weather  and  Radar  Processor  (WARP) ,  the  Integrated 
Terminal  Weather  System  (ITWS) ,  and  into  commercial  vendor 
supplied  weather  systems  under  Cooperative  Research  and 
Development  Agreements  (CRDAs) .  The  AWR  graphical  weather 
products  are  the  focus  of  this  evaluation  report. 

1.1  PURPOSE. 

The  purpose  of  this  report  is  to  document  results  of  the  AWR 
graphical  weather  products  evaluation.  The  evaluation  was 
conducted  from  human  factors,  meteorological,  and  system 
administration  perspectives.  Since  the  developer  (NCAR)  was  not 
bound  by  FAA  design  requirements,  the  recommendations  provided 
herein  are  intended  as  feedback  for  the  development  of  future 
graphical  weather  products . 

1.2  SCOPE. 

For  the  remainder  of  this  report,  the  AWR  graphical  weather 
products  will  be  referred  to  as  the  Aviation  Weather  Research 
Products  (AWRP) .  It  is  important  to  note  that  the  AWRP  is  not  a 
system,  but  a  collection  of  products.  This  report  summarizes  the 
evaluation  of  the  weather  products  individually,  as  well  as 
collectively.  The  report  is  written  using  FAA-STD-024B  as  a 
guideline.  The  contents  of  this  report  include  a  summary  of  the 
AWRP  configuration,  evaluation  descriptions,  results. 
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conclusions,  and  finally,  recommendations  for  the  AWRP  and 
graphical  weather  systems  in  general. 

REFERENCE  DOCUMENTS. 


FAA-STD-024B  FAA  Standard  024B,  Content  and  Format 

Requirements  for  the  Preparation  of  Test 
and  Evaluation  Documentation,  August  22, 
1994. 

ESD-TR-86-278  Guidelines  for  Designing  User  Interface 

Software,  August  1986. 

AFOTEC  Pamphlet  99-102  Air  Force  Operational  Test  and 

Evaluation  Center  (AFOTEC)  Software 
Usability  Evaluation  Guide,  Volume  4, 
June  1994 . 


MIL-STD-1472D 

FAA  7210.3 

FAA  7110.10 
FAA  7110.65 
FAA  ORDER  7032.15 

FCM-P27-1992 


Military  Standard  1472D,  Human 
Engineering  Guidelines,  March  14,  1989. 

Facility  Operation  and  Administration, 
September  16,  1993. 

Flight  Service,  September  16,  1993. 

Air  Traffic  Control,  September  16,  1993. 

Air  Traffic  Weather  Needs  and 
Requirements  Report,  October  5,  1994. 

National  Aviation  Weather  Program  Plan, 
September,  1992. 

FAA  ACT-320  1994  Job  Task  Analysis  (JTA) 
Report,  March  1995,  DRAFT. 


1993  FAA  Technical  Center  Summer 
Demonstration  Report  of  the  AWPG, 
October  20,  1993. 

AWRP  Documentation  --  AWPG  User's  Guide, 
March  1995 . 
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AWRP  Documentation  --  AWRP  Installation 
Executable  Only,  March  1995. 

AWRP  Documentation  --  AWRP  Installation 
Source  Code,  March  1995. 

AWRP  Documentation  --  AWRP  Live  System 
Management,  March  1995. 


3.  SYSTEM  DESCRIPTION. 

3.1  MISSION  REVIEW. 

The  AWRP  is  not  part  of  an  FAA  acquisition  program.  The 
evaluated  displays  are  the  result  of  aviation  weather  related 
research  coupled  with  user  identified  needs.  Concepts  and 
techniques  of  the  AWRP  identified  as  having  utility  should  be 
considered  for  incorporation  into  existing  and  future  graphical 
weather  systems,  such  as  WARP  and  ITWS  so  that  they  might  better 
satisfy  their  respective  mission  requirements. 

3.2  SYSTEM  CONFIGURATION. 

The  platform  used  for  the  display  of  products  consisted  of  AWRP 
software,  a  Sun  Sparc20  Workstation,  a  Cisco  router,  and  a  Data 
Service  Unit/Channel  Service  Unit  (DSU/CSU)  connected  to  a 
56-kilobytes  (Kb)  per  second  dedicated  line.  Computer  forecast 
data,  NWS  Family  of  Services  (FOS) ,  and  vendor  supplied  radar 
mosaic  and  lightning  data  are  collected  at  NCAR  and  forwarded  to 
the  FAA  Technical  Center  via  the  dedicated  line.  The  AWRP 
software  converts  the  data  into  graphical  and  textual  weather 
products . 

3.3  INTERFACES. 

Not  applicable. 

4.  EV?lLUATION  DESCRIPTION. 

4.1  SCHFDTTT.F  AND  LOCATIONS. 

The  AWRP  evaluation  was  performed  from  July  17  to  August  25, 
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1995,  in  a  nonoperational  environment  at  the  FAA  Technical  Center 
Aviation  Weather  Facility.  In  addition,  informal  evaluations  and 
troubleshooting  were  conducted  on  an  on-going  basis  since  the 
March  1995  release  of  the  software. 

4.2  PARTICIPANTS. 

Personnel  from  the  following  organizations  conducted  and 
supported  the  AWRP  evaluation: 

Organization  Role 

Test  Director  and  Evaluators 
Technical  Support 

4.3  SPECIALIZED  EVALUATION  EQUIPMENT. 

No  specialized  test  equipment  was  required. 

4.4  EVALUATION  OBJECTIVES  AND  CRITERIA. 

4.4.1  Human  Factors  Objectives. . 

The  objectives  of  the  human  factors  evaluation  were  as  follows: 

a.  To  assess  the  AWRP  user-system  interface; 

b.  To  assess  the  AWRP  User's  Guide;  and 

c.  To  determine  if  the  AWRP  meets  the  job  task  and  weather 
information  needs  of  controllers  and  specialists. 

4.4.2  Human  Factors  Evaluation  Criteria. 

Human  factors  issues  associated  with  the  AWRP  were  determined  by 
an  independent  Human  Factors  Specialist  familiar  with  air  traffic 
functions.  Criteria  for  the  human  factors  aspects  of  the 
evaluation  are  discussed  in  the  following  paragraphs. 

4. 4. 2.1  User-System  Interface  Guidelines. 

Information  from  human  factors  standards  and  interface  documents 
(i.e..  Guidelines  for  Designing  User  Interface  Software,  AFOTEC 


ACT- 320 
NCAR 


5 


Software  Usability  Evaluation  Guide,  and  Military  Standard  1472D) 
were  used  to  guide  the  evaluation  of  the  user-system  interface; 
specifically,  screen  layout,  menu  structure,  color  coding, 
symbology,  and  data  entry.  The  success  criteria  for  this  portion 
of  the  evaluation  were  in  general  agreement  with  the  guidelines 
listed  in  paragraph  5. 1.1.1. 

The  AWRP  was  not  designed  specifically  to  these  guidelines. 
However,  the  guidelines  are  industry  accepted  standards  that 
characterize  effective  user-interface  design. 

4. 4. 2. 2  AWRP  User's  Guide. 


The  AWRP  User's  Guide  was  assessed  according  to  its  accuracy 
regarding  system  performance  and  its  effectiveness  in  describing 
product  use .  The  success  criteria  were  accurate  explanation  of 
interface  devices,  satisfactory  product  and  function  operation, 
and  appropriate  error  messages . 

4. 4. 2. 3  Job  Task  and  Weather  Information  Needs. 

Job  task  and  weather  information  needs  determined  from  the  JTA, 
ATUWG  meetings,  and  the  1993  Demval  were  used  for  comparisons  to 
assess  the  extent  to  which  the  AWRP  met  user  needs.  The  success 
criteria  for  this  assessment  were  satisfaction  of  the  task  and 
information  needs  as  identified  in  the  JTA,  ATUWG  meetings,  and 
the  1993  Demval. 

4.4.3  Meteorological  Objectives. 

The  objectives  of  the  meteorological  evaluation  were  as  follows: 

a.  To  determine  the  amount  of  meteorological 
interpretation  needed  for  the  individual  products; 

b.  To  determine  whether,  and  to  what  extent,  the  products 
show  improvement  over  existing  operational  products; 
and 

c.  To  determine  the  meteorological  consistency  both  within 
and  between  products. 
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4.4.4  Meteorological  Evaluation  Criteria. 

The  meteorological  evaluation  was  conducted  by  an  independent 
meteorologist  familiar  with  air  traffic  functions.  A  product-by- 
product  assessment  was  performed  that  addressed  whether  too  much 
or  too  little  information  was  presented  for  air  traffic 
personnel,  the  meteorological  symbolism  used  in  the  products,  and 
compliance  with  generally  accepted  meteorological  practices. 

Improvements  over  existing  products  were  determined  by  reviewing 
a  subset  of  current  AFSS  weather  products  and  their  use,  then 
directly  comparing  the  AWRP  to  the  AFSS  products.  Current  AFSS 
weather  products  were  reviewed  at  the  Millville,  New  Jersey,  AFSS 
prior  to  the  official  start  of  the  AWRP  evaluation. 

Meteorological  consistency  was  determined  by  comparison  of 
graphical  and  textual  portions  of  the  AWRP  to  ensure  that 
graphics  agree  with  accompanying  text.  Consistency  was  also 
determined  by  comparison  of  products  showing  related  phenomena 
over  the  same  geographical  region.  Guidelines  for  meteorological 
consistency  are  listed  in  paragraph  5.2.3. 

The  meteorological  success  criteria  were: 

a.  Limited  or  no  meteorological  interpretation  needed; 

b.  Improvement  over  current  operational  AFSS  weather 
products;  and 

c.  Compliance  with  the  guidelines  in  paragraph  5 .2 .3 . 

4.4.5  System  Administrator  Ob-iectives. 

The  objectives  of  the  system  administration  evaluation  were  to 
determine  if  the  AWRP  installation  and  system  management 
documents  were  usable  and  effectively  described  installation  and 
system  management  procedures . 

4.4.6  Evaluation  Criteria. 

The  AWRP  installation  and  system  management  documents  were 
assessed  according  to  their  accuracy  and  effectiveness  regarding 
installation  and  system  management  procedures. 
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4.5  EVALUATION  DESCRIPTION. 

4.5.1  Human  Factors  Evaluation  Description. 

The  human  factors  evaluation  was  conducted  in  two  stages.  The 
first  consisted  of  a  general  review  based  on  user-system 
interface  standards  and  product  improvements  which  were 
incorporated  following  the  1993  Demval.  The  second  stage  was  a 
more  specific  analysis  based  on  user  needs,  job  tasks,  and  how 
products  would  work  in  an  operational  setting.  Each  evaluation 
stage  is  discussed  below. 

4. 5.1.1  General  Review  Stage. 

This  stage  of  the  evaluation  compared  product  and  user- system 
interface  characteristics  with  the  guidelines  listed  in  paragraph 
5.1.1.  The  guidelines  focus  on  the  use  of  the  interface  in  the 
context  of  user  tasks,  display  concepts  (e.g.,  color  coding, 
icons,  and  functionality),  and  information  format  (e.g., 
interpretability,  and  workload) .  Each  product  was  evaluated  in 
order  to  determine  how  well  the  interface  and  display  concepts 
met  the  guidelines. 

An  additional  component  of  the  general  review  stage  was  the 
assessment  of  the  User's  Guide.  The  evaluator  determined  if  the 
following  areas  were  described  accurately  and  concisely: 

a.  Interface  devices  (e.g.,  mouse  and  trackball); 

b.  Product  operation  and  output;  and 

c.  Function  (e.g.,  zoom,  pan,  text,  route  select,  etc...) 
operation  and  output. 

The  User's  Guide  was  also  reviewed  to  ensure  that  error  messages 
were  defined  and  possible  solutions  were  outlined. 

4. 5. 1.2  Operational  Analysis  Stage. 

4 . 5 . 1 . 2 . 1  JTA  Background . 

The  JTA  report  was  developed  internally  by  ACT-320  in  an  effort 
to  better  understand  the  weather  requirements  of  Traffic 
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Management  Coordinators  at  ARTCC  TMUs  and  AFSS  Specialists.  This 
information  was  to  be  provided  to  NCAR  as  feedback  for  the 
development  of  AWR  graphical  weather  products .  However,  AWRP 
developers  (NCAR)  were  not  required  to  utilize  JTA  information 
and  recommendations  iri  their  design  of  the  AWRP. 

The  JTA  was  developed  in  two  stages.  During  the  first  stage,  a 
research  effort  was  conducted  in  which  several  reference  reports 
(FAA  7210.3,  FAA  7110.10,  FAA  7110.65,  Air  Traffic  Weather  Needs 
and  Requirements  Report,  and  National  Aviation  Weather  Program 
Plan)  were  used  to  develop  task  lists  for  AFSS  Specialists  and 
TMCs.  A  list  of  required  information  was  developed  for  any  task 
involving  weather  assessment  and  dissemination. 

Following  the  compilation  of  job  task  lists  and  weather 
information  requirements,  job  tasks  were  depicted  using  flow 
charts.  Corresponding  weather  information  requirements  were 
outlined  in  tables.  Weather  tables  were  then  further  expanded  to 
include  the  process  by  which  the  required  information  was 
obtained.  Process  information  included  sources,  interpretation, 
format,  and  utility  of  the  weather  data  associated  with  the  job 
tasks . 

The  second  stage  of  the  JTA  involved  confirming  all  information 
compiled  during  the  first  stage.  Confirmation  was  accomplished 
by  comparing  information  contained  in  the  flowcharts  and  tables 
to  observe  job  task  and  weather  information  requirements  in  an 
operational  setting. 

4 . 5 . 1 . 2 . 2  ATUWG  and  1993  Demval  Background . 

During  the  development  of  the  AWRP,  the  ATUWG  provided 
operationally  based  feedback  on  AWRP.  Desired  product 
improvements,  as  determined  from  the  ATUWG  meetings  and  the  1993 
Demval,  were  used  as  representative  examples  of  features  needed 
to  provide  or  improve  operational  utility. 

4 ■ 5 ■ 1 ■ 2 . 3  Evaluation  Approach . 

During  the  operational  analysis,  the  AWRP  was  compared  to  user 
requests  from  the  ATUWG,  the  1993  Demval  results,  and 
recommendations  from  the  JTA  report.  Compliance  with  user 
requests  and  recommendations  was  scored  using  a  binary  approach 
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(i.e.,  request /recommendation  met  or  not  met).  In  addition,  user 
needs  not  addressed  at  all  by  the  AWRP  were  identified.  The 
results  of  the  comparison  provide  insight  to  the  probable 
operational  utility  of  the  individual  products.  Requests  from 
the  ATUWG  and  1993  Demval  are  discussed  in  paragraph  5. 1.2.1. 
Information  requirements  for  AFSS  Specialists  and  ARTCC  TMCs  are 
discussed  in  paragraph  5. 1.2. 2.  Complete  JTA  flowcharts  and 
weather  information  tables  are  listed  in  appendix  A. 

■4  ..5.2  Meteorological  Evaluation  Description. 

4^.._5.^..2 . 1  Meteorological  Interpretation .  i. 

Meteorological  interpretation  was  determined  by  observing  the 
AWRP  and  determining  whether  the  products  were  clearly 
understandable  or  if  interpretation  was  needed.  The  level  of 
interpretation  was  rated  on  a  numerical  scale  of  1  to  10  with  1 
requiring  no  interpretation  and  10  requiring  much  interpretation. 

4. 5. 2. 2  Improvement  Over  Current  Products. 

Improvement  over  current  AFSS  weather  products  was  determined  by 
comparing  the  AWRP  to  existing  operational  products.  .  The 
comparison  was  not  intended  as  a  verification  of  the  products  or 
the  weather  data  used  as  input;  rather,  the  comparison 
concentrated  on  the  format,  timeliness,  and  amount  of 
meteorological  information  presented.  Existing  operational 
products  were  viewed  at  the  Millville,  New  Jersey,  AFSS.  While 
products  may  vary  between  AFSSs,  the  products  reviewed  at  the 
Millville  AFSS  are  considered  representative  of  current 
operational  capabilities.  After  a  thorough  review  of  current 
AFSS  weather  products,  the  AWRP  was  assessed  and  the  level  of 
improvement  contributed  by  the  AWRP  was  determined. 

4. 5. 2. 3  -M.eteorolo_qical  Consistency. 

The  AWRP  was  evaluated  for  meteorological  consistency  between 
graphics  and  text.  In  addition,  consistency  between  individual 
products  showing  related  phenomena  over  the  same  geographical 
region  was  evaluated.  Consistency  was  evaluated  in  order  to 
ensure  that  conflicting  information  was  not  presented  within  and 
between  products.  Consistency  was  scored  using  a  binary  method 
(i.e.,  consistency  met  or  not  met) . 
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4.5.3  System  Administrator  Evaluation  Description. 

The  following  AWRP  documents  were  evaluated  to  ensure  accuracy 
and  ease  of  understanding  of  procedures  and  information: 

a.  AWRP  Installation  -  Executable  Code, 

b.  AWRP  Installation  -  Source  Code,  and 

c.  AWRP  Live  System  Management. 

Using  the  vendor  supplied  documentation,  the  system  administrator 
performed  system  installation  and  operation. 

4.6  DATA  COLLECTION  AND  ANALYSIS  METHOD. 

During  the  human  factors  portion  of  the  evaluation,  data  was 
recorded  and  summarized  using  guidelines  and  standards  tables. 
Problems  with  the  User's  Guide  and  comparisons  to  the  JTA  were 
noted  and  summarized  as  well.  For  the  meteorological  portion  of 
the  evaluation,  interpretation  issues  were  recorded  using  a  10- 
point  scale  and  summarized. 

5.  RESULTS  AND  DISCUSSION. 

5.1  HUMAN  FACTORS  ■EVALUATION  RESULTS  AND  DISCUSSION. 

This  section  will  present  the  results  of  the  human  factors 
evaluation.  The  evaluation  was  conducted  in  two  stages; 
therefore,  results  will  be  discussed  separately  for  each  stage. 

5.1.1  General  Review  Stage . 

This  section  will  present  the  results  from  the  general  review 
portion  of  the  evaluation.  Specifically,  it  will  discuss  the 
comparison  of  AWR  product  and  user-system  interface 
characteristics  to  human  factors  standards  and  guidelines.  In 
addition,  the  assessment  of  the  User's  Guide  will  be  discussed. 

5. 1.1.1  User-System  Interface  Guidelines. 

The  AWRP  interface  was  evaluated  using  Guidelines  for  Designing 
TTsar  Interface  Software.  AFOTEC  Software  Usability  Evaluation 
Guide,  and  MIL- STD  1472D.  A  subset  of  interface  design 
guidelines  directly  relating  to  the  AWRP  interface  were 
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identified  from  these  references.  AWRP  interface  characteristics 
were  then  compared  to  these  guidelines.  The  results  from  these 
comparisons  are  discussed  in  the  following  paragraphs. 

5. 1.1. 1.1  Guidelines  for  Designing  User  Interface  Software. 

Table  1  presents  the  results  of  the  comparison  of  the  AWRP 
interface  characteristics  to  design  guidelines  from  Guidelines 
£o.r__De signing  User  Interface  Software. 

Design  guidelines  stated  in  table  1  and  not  met  by  the  AWRP  are 
as  follows : 

a.  Guideline  4:  Standard  labeling  is  not  currently  used  on 
the  AWRP  display.  Product  buttons  are  labeled  using  Unix- style 
language  (e.g.,  Flt_Cat  and  Frz_Level)  as  opposed  to  full  product 
names.  Additionally,  some  labels  are  abbreviated  (e.g..  Show 
Vert  and  Posloc) .  Buttons  should  be  labeled  in. standard  language 
using  as  few  abbreviations  as  possible. 

b.  Guideline  5;  Airport  labels  on  the  Flight  Category 
product  are  considered  critical  data  especially  in  congested 
areas.  As  a  user  zooms  in,  the  flight  category  dots  obscure  the 
identifiers  making  it  difficult  to  accurately  identify  the 
stations.  Figure  1  illustrates  this  problem.  Airport 
identifiers  should  be  highlighted  so  stations  can  be  identified. 

c.  Guideline  6:  A  definition  index  for  flight  categories  is 
not  available  on  the  AWRP  display.  Additionally,  an  obscuration 
symbol  index  is  not  available.  This  information  should  be 
included  in  the  on-line  help  system.  Provide  a  definition  index 
in  the  on-line  help  system  for  both  flight  categories  and 
obscuration  symbols.  Also,  provide  obscuration  symbol  index  in 
the  User's  Guide. 

d.  Guideline  11:  Looping  or  animation  is  not  available  for 
dynamic  products  (e.g..  Radar  Mosaic  and  Lightning).  It  is 
difficult  to  effectively  use  dynamic  products  without  the  benefit 
of  trend  information  that  is  usually  evident  during  product 
animation.  Looping  should  be  available  for  dynamic  products. 
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TABLE  1.  GUIDELINES  FOR  DESIGNING  USER  INTERFACE  SOFTWARE 


GUIDELINE 

MET? 

1.  Use  graphics  for  spatial 
and  temporal  information. 

Yes 

2 .  Use  graphics  for  quick 
scanning  and  data 
comparisons . 

Yes 

3 .  Use  graphics  when  users 

Yes 

must  monitor  changing  data. 

• 

4 .  Use  standard  format  and 
labeling  for  all  graphic 
products . 

No 

5.  Highlight  features  that 
show  critical  data. 

No 

6 .  Include  reference  index 
for  data  comparisons  to 
critical  values. 

No 

7.  Annotate  graphics  with 
text  when  precise  readings 

Yes 

are  necessary. 

8 .  Format  annotations 
consistently  in  relation  to 
graphic  elements . 

Yes 

9 .  Display  all  labels  in 
normal  reading  orientation. 

Yes 

10.  Design  icons  to  resemble 
objects  or  processes  they 
represent . 

Yes 

11.  Use  animation  for  dynamic 
information  (i.e.,  looping). 

No 

■  12.  Establish  standard 

■  imeanings  for  symbols. 

Yes 

Unix- style  language  should 
not  be  used  to  label 
buttons  (Flt_Cat) . 


Flight  Category  dots 
obscure  airport  text . 


No  reference  is  available 
for  Flight  Categories  or 
visibility  obstruction. 


Good  examples :  Winds 
Aloft,  Temperature,  Area 
and  Cell  Motion. 


Looping  is  not  available 


Good  Examples:  Flight 
Category,  Wind  Barbs. 
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FIGURE  1.  OBSCURATION  OF  STATION  IDENTIFIER  IN  FLIGHT  CATEGORY  PRODUCT 


B.1.1.1.2  AFOTEC  Software  Usability  Evaluation  Guide.- 

Table  2  presents  the  results  of  the  comparison  of  the  AWRP 
interface  characteristics  to  a  subset  of  AFOTEC  design 
guidelines. 

Design  guidelines  stated  in  table  2  not  met  by  the  AWRP  are  as 
follows: 

a.  Guideline  1:  In  the  Positional  Locator  mode,  data 
entry  requires  a  fixed  fonnat;  however,  the  system  does  not 
prompt  for  this  fixed  format.  If  the  user  enters  the  data 
incorrectly,  the  system  only  beeps  and  offers  no  indication  as  to 
why  the  entry  is  wrong.  The  Positional  Locator  window  should 
provide  a  fixed  field  in  which  data  is  entered.  If  data  is 
entered  incorrectly,  a  message  box  should  prompt  the  user 
regarding  the  correct  format  for  data  entry. 

b.  Guideline  6:  Data  entered  in  the  Positional  Locator 
window  is  very  difficult  to  edit.  While  in  the  hollow  box,  the 
backspace  or  delete  keys  do  not  work  to  fix  the  entry.  Editing 
data  should  be  simple.  Instructions  regarding  data  editing 
should  appear  in  both  on-line  help  and  the  User's  Guide. 

c.  Guideline  8:  While  the  system  does  have  a  busy 
indicator,  it  does  not  always  appear  immediately  following  user 
inputs.  In  the  text  mode,  the  busy  indicator  appears  several 
seconds  after  a  request .  Since  the  user  thinks  the  input  did  not 
register,  several  more  clicks  may  be  entered  until  feedback  is 
received.  If  the  user  has  entered  more  than  one  request,  several 
copies  of  the  requested  text  will  appear  in  the  window.  The  busy 
indicator  should  appear  immediately  following  user  input. 

d.  ■  Guideline  9:  The  AWRP  has  no  error  messages.  For 
instance,  in  the  Positional  Locator  mode,  if  the  user  enters  data 
incorrectly,  the  system  simply  beeps.  The  user  does  not  know  why 
the  system  beeped  or  how  to  fix  the  problem.  Error  messages 
should  be  incorporated  which  clearly  identify  the  problem  and  a 
potential  solution. 
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TABLE  2.  AFOTEC  SYSTEM  USABILITY  GUIDELINES 


GUIDELINE 

MET? 

1.  System  prompts  when 
fixed  or  maximum  data 
entry  is  required. 

No 

2 .  Cursor  is  easy  to 
locate . 

Yes 

3 .  Data  fields  are 
adequately  labeled. 

Yes 

4.  Data  entry  is  user- 
paced. 

Yes 

5 .  System  minimizes 
keyboard  and  keypad  data 
entry. 

Yes 

6.  Text  inputs  are  easy  to 
edit . 

No 

7 .  The  user  is  not 
required  to  enter  data 
already  available  to 
system. 

Yes 

8.  System  provides  quick, 
positive  feedback  after 
inputs . 

No 

9 .  Error  messages  are 
clear  and  succinct. 

No 

10.  It  is  clear  what  mode 
the  user  is  in. 

No 

Data  entry  in  the  Positional 
Locator  mode  allows  user  to 
enter  data  in  any  format ; 
yet,  data  must  be  entered  in 
XXX/XXX/XXX . 


Data  entry  for  Positional 
Locator  is  difficult  to 
edit . 


Busy  indicator  does  not 
immediately  follow  text 
input . 


AWRP  has  no  error  messages . 


Modes  do  not  always 
disengage.  Two  mode  buttons 
can  be  depressed. 
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TABLE  2.  AFOTEC  SYSTEM  USABILITY  GUIDELINES  (Continued) 


6UIDLINE 

MET? 

11 .  Menus  and  menu 
selection  options  are 
appropriately  labeled. 

Yes 

12.  System  defaults  are 
set  on  the  most  frequently 
used  options . 

No 

13.  A  processing  or 
working  message  appears 
while  system  is  working. 

No 

14 .  Menu  selection  options 
are  organized  by  function 
and  order. 

No 

15.  Menu  options  are 
consistent  in  wording, 
order,  and  format. 

Yes 

16.  Wording  of  menu 
options  is  consistent  with 
functions  they  control. 

Yes 

17.  Menu  options  are  easy 
to  locate  with  menu 
hierarchy.  • 

Yes 

18.  Inactive  menu 
selections  de-emphasized 
on  display. 

Yes 

19.  It  is  easy  to  move 
along  different  levels  of 

menu . 

Yes 

COMMENTS 


AWRP  defaults  to  Icing 
Product.  Airtnets  or  Radar 
screen  would  be  more 
appropriate . 


Busy  indicator  does  not 
accurately  reflect  system 
state. 


Mouse  modes  should  be 
organized  according  to  most 
frequently  used.  Zoom  and 
pan  would  most  likely  be 
most  frequently  used. 


TABLE  2.  AFOTEC  SYSTEM  USABILITY  GUIDELINES  (Continued) 


COMMENTS 


GUIDLINE 

MET? 

20.  The  system  minimizes 
shifting  from  mouse  to 
keyboard  input . 

Yes 

21.  The  amount  of  data 
presented  on  the  screen  is 
appropriate . 

Yes 

22 .  Data  is  displayed  in  a 
logical  manner. 

Yes 

23 .  In  any  display,  all 
necessary  data  are 
display. 

Yes 

1  24 .  Display  update  is  fast 
enough  to  keep  up  with 
user  inputs. 

No 

25.  Display  minimizes  need 
for  interpretation  or 
memorization. 

Yes 

26.  Activation  of  menu 
options  is  easy. 

Yes 

27.  Display  formats  are 
consistent  across  system. 

Yes 

28.  Wording  is  consistent 
across  all  displays. 

Yes 

29.  Text  displays  are  easy 
to  read. 

No 

30.  Display  coding  methods 
consistent  across 
displays. 

Yes 

It  is  controlled  by  the 
number  of  overlays  utilized 
by  the  user. 


For  operational  setting, 
text  is  not  displayed  fast 
enough . 


Virtually  no  memorization  is 
required. 


Font  size  should  be  larger, 
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TABLE  2.  AFOTEC  SYSTEM  USABILITY  GUIDELINES  (Continued) 


GUIDELINE 

MET? 

COMMENTS 

31.  Windows  are  easy  to 
open  and  close. 

No 

Given  prototyping 
environment,  system  crashes 
by  accidentally  closing 
windows . 

32.  It  is  easy  to  scroll 
with  windows. 

No 

It  is  sometimes  difficult  to 
operate  the  altitude  slider. 

It  is  also  very  slow. 

33.  Display  symbols 
conform  to  operational 
conventions . 

Yes 

34 .  Graphic  symbology  is 
appropriate  for  the  info 
it  represents. 

Yes 

35.  Icons  used 
consistently  across 
different  displays. 

Yes 

36.  The  use  of  color 
enhances  display 
readability. 

No 

Need  better  contrast  between 
wind  barbs /wind  contours  and 

1 ightning/radar . 

37 .  System  messages  are 
informative  and  concise. 

No 

System  messages  do  not 
exist . 

38.  User's  manual  provides 
complete  descriptions  of 
procedures. 

No 

See  section  5. 1.1. 2. 

39.  User's  manual  easy  to 

use. 

No 

See  section  5. 1.1. 2. 

40.  Display  colors  are 
consistent  with 
operational  conventions. 

Yes 

No  standard  colors  exist  for 
Flight  Category,  AIRMETs, 

Icing,  or  Turbulence. 
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TABLE  2.  AFOTEC  SYSTEM  USABILITY  GUIDELINES  (Continued) 


GUIDELINE 

MET? 

COMMENTS 

41.  System  functions  are 
organized  in  a  manner  that 
is  consistent  with  job 
tasks . 

No 

See  section  5. 1.2. 2. 2. 

42 ,  Routine  operations  can 
be  performed  w/out  user ' s 
manual . 

Yes 

43.  System  allows  operator 
functions  to  be  performed 
optimally. 

No 

See  section  5. 1.2. 2. 2. 

44 .  Functions  that  the 
user  has  not  performed  for 
a  period  of  time  are 
easily  relearned. 

Yes 

e.  Guideline  10:  The  modes  on  the  display  do  not  always 
disengage.  For  instance,  when  changing  modes,  both  the  old  and 
new  mode  buttons  remain  depressed.  When  this  happens,  the  user 
must  click  on  the  new  mode  button  a  second  time  to  disengage  the 
previous  mode.  All  mode  buttons  should  easily  disengage  when  a 
second  mode  button  is  pressed. 

f.  Guideline  12:  System  defaults  should  be  set  to  the 
most  frequently  used  products;  however,  the  AWRP  currently 
defaults  to  the  Icing  Product.  If  both  AFSS  and  TMU  users  are 
considered,  the  Radar  Mosaic  and  Airmen's  Meteorological 
Information  (AIRMET) /Significant  Meteorological  Information 
(SIGMET)  products  are  probably  most  often  used.  The  system 
should  default  to  one  of  these  products  or  allow  user  definition 
of  defaults. 

g.  Guideline  13:  As  indicated  earlier  (item  c) ,  the  AWRP 
display  has  a  busy  indicator;  however,  it  does  not  accurately 
reflect  the  system  state. 
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h.  Guideline  14:  Menu  items  are  not  organized  by  function 
or  order.  The  mouse  mode  and  product  buttons  should  be  organized 
according  to  use.  For  instance,  the  zoom  button  should  be  first 
in  the  row  of  mode  buttons.  The  zoom  mode  is  used  most  of  ten; 
yet,  it  is  located  in  the  middle  of  the  buttons.  It  can  be 
difficult  to  locate  in  time  critical  situations.  Users  have 
indicated  that  convective  products  are  most  important  and  most 
often  needed;  yet,  the  radar  button  is  not  first  in  the  product 
list.  Additionally,  overlays  in  the  Weather  Overlay  menu  are  not 
organized  according  to  function.  For  instance,  when  obtaining 
radar  overlays,  convective  related  overlays  (e.g..  Lightning, 

Area  Motions,  and  Convective  SIGMETS)  are  scattered  throughout 
the  list.  Overlays  should  be  grouped  by  product  or  function. 

i.  Guideline  24:  In  some  instances,  display  updates  do 
not  keep  pace  with  user  inputs.  When  retrieving  text,  the  system 
requires  several  seconds  to  display  the  text.  Additionally,  when 
changing  flight  levels,  the  display  requires  several  seconds  to 
update . 

j.  Guideline  29:  Font  size  should  be  larger.  The  display 
may  be  difficult  to  read  at  distances  greater  than  2  feet. 

k.  Guideline  31:  Windows  are  difficult  to  close.  While 
all  windows  have  exit  buttons,  they  also  have  the  "close  dot"  in 
the  upper  left  hand  corner.  Most  users  are  familiar  with  the 
Personal  Computer  (PC)  windows  environment  where  pressing  the  dot 
closes  the  window.  However,  in  a  Unix  environment,  pressing  the 
dot  exits  the  window  and  causes  the  system  to  crash. 

l.  Guideline  32:  The  altitude  slider  bar  Can  be  difficult 
to  use.  Not  only  is  the  update  rate  very  slow,  but  the  slider 
bar  can  be  difficult  to  manipulate.  Many  times  when  releasing 
the  "thumb"  or  slider  button  at  a  new  altitude,  the  "thumb"  or 
slider  button  has  continued  to  move  higher  or  lower.  Given  the 
slow  update  rate,  the  user  must  wait  for  the  altitude  to  adjust 
and  then  move  the  thumb  to  the  correct  altitude. 

m.  Guideline  36:  Color  coding  does  not  enhance  display 
readability  in  two  instances.  Better  contrast  needs  to  be 
provided  between  the  wind  barbs  and  wind  contours  at  70  and  130 
knots  (kts) (see  figure  2).  Lightning  strikes  are  also  difficult 
to  detect  against  level  3  and  4  precipitation  (see  figure  3). 
Colors  need  to  be  changed  to  provide  better  contrast. 
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n.  Guideline  37:  The  AWRP  display  does  not  have  system 
messages .  Informative  and  concise  system  messages  regarding 
errors  or  incorrect  data  entry  should  be  added. 

5. 1.1. 1.3  Military  Standard  1472D. 

Table  3  presents  the  results  of  the  comparison  of  the  AWRP 
interface  characteristics  to  Military  Standard  1472D  (MIL-STD 
1472D)  standards.  Design  guidelines  stated  in  table  3  not  met  by 
the  AWRP  are  as  follows : 

a.  Guideline  1:  Auditoary  or  visual  warnings  and/or  alerts 
are  not  provided  on  the  AWRP  display.  Warnings  should  be 
implemented  for  critical  changes  in  data  and  data  entry  errors. 

b.  Guideline  2:  Sufficient  contrast  is  not  provided  between 
wind  barbs  and  70  or  130  kts  wind  contours  (see  figure  2) ; 
lightning  strikes  and  levels  3  and  4  of  the  Radar  Mosaic  (see 
figure  3) ;  and  the  Icing  AIRMET  and  Mountain  Obscuration  AIRMET 
(see  figure  4) .  Colors  should  be  changed  to  provide  appropriate 
contrast . 

c.  Guideline  12:  In  some  instances,  the  display  is  not 
refreshed  in  a  synchronous  manner.  Radar  Mosaic  provides 
seamless  updates;  however,  Aviation  Gridded  Forecast  System 
(AGFS)  products  require  2  to  3  seconds  to  refresh.  During  these 
3  seconds,  the  screen  goes  blank,  leaving  the  controller  with  no 
data  to  refer  to.  Refresh  rates  need  to  be  faster  for  AGFS 
products . 

d.  Guideline  14:  The  font  style  is  appropriate;  however, 
the  font  size  should  be  larger  to  make  character  discrimination 
easier. 

5. 1.1. 2  User's  Guide. 

The  AWRP  User's  Guide  was  evaluated  for  accuracy  regarding  system 
performance  and  its  effectiveness  in  describing  product  use. 

More  specifically,  the  User's  Guide  was  reviewed  for  an  accurate 
explanation  of  interface  devices,  product  and  function  operation, 
and  error  messages.  Paragraphs  -5 . 1 . 1 . 2 . 1  and  5. 1.1. 2. 2  contain 
comments  regarding  the  User's  Guide. 
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FIGURE  2.  WINDS  ALOFT  COLOR  CONTRAST 
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FIGURE  3.  LIGHTNING  STRIKE  COLOR  CONTRAST 


TABLE  3.  MIL-STD-1472D  STANDARDS 


STANDARD 

MET? 

COMMENTS 

1.  Alert /warning  display  shall 
provide  increased  probability 
of  detecting  condition. 

No 

Warnings  or  alerts  are 
not  provided. 

2.  Sufficient  contrast  shall 
be  provided  between  display 
info  and  display  background. 

No 

Green  wind  barbs  do  not 
contrast  with  70  or  130 
kts  contours. 

Lightning  does  not 
contrast  with  VIP 
levels.  Icing  and  Mtn. 
Obscuration  AIRMETs  are 
hard  to  differentiate. 

3.  Information  shall  be 
sufficient  to  allow  the  user 
to  perform  intended  mission. 

No 

See  section  5.1.2. 

4.  Redundancy  in  the  display 
shall  be  avoided  unless 
required  to  achieve 
reliability. 

Yes 

5 .  Company  names  not  related 
to  function  shall  not  be  on 
the  panel . 

Yes 

6.  Displayed  information  shall 
have  duration  of  sufficient 
length  to  be  reliably 
detected. 

Yes 

User-paced  product  by 
product .  User  controls 
information  rate. 

7.  Numerics  not  used  when 
perception  of  pattern 
variations  is  important. 

Yes 

Wind  barbs  show  overall 
wind  direction. 

8 .  Red  is  used  to  alert . 

Yes 

Level  6  precip.,  LIFR. 

■ 

9.  Green  denotes  satisfactory 
conditions. 

Yes 
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TABLE  3.  MIL-STD-1472D  STANDARDS  (Continued) 


STANDARD 

MET? 

COMMENTS 

10.  White  implies  no 
decision. 

Yes 

11.  Yellow  advises  caution 
or  marginal  conditions. 

Yes 

12 .  Displays  requiring 
refreshed  information  shall 
update  in  a  synchronous 
manner. 

No 

Radar  provides  seamless 
updating;  AGFS  products 
require  2-3  second 
refresh  rate. 

13 .  Avoid  use  of  flashing 
lights . 

Yes 

14.  Font  style  should  allow 
discrimination  of  similar 
characters  such  as  1  from  1 . 

No 

Font  style  is 
appropriate.  Font  size 
should  be  larger. 
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FIGURE  4.  ICING  AND  MOUNTAIN  OBSCURATION  AIRMET  COLOR  CONTRAST 


5. 1.1. 2.1  General  Comments. 


Overall,  the  User's  Guide  provided  a  good  description  of  AWR 
products.  However,  instructions  regarding  manipulation  of  the 
user  interface  in  section  3.  of  the  User's  Guide  tended  to  be 
inconsistent,  unclear,  and  inaccurate  at  times.  Paragraph 

5. 1.1. 2. 2  will  address  specific  comments  regarding  the  entire 
document . 

5. 1.1. 2. 2  Specific  Comments. 

All  references  (e.g.,  page  2,  section  1.2)  in  this  paragraph 
refer  to  the  AWRP  User's  Guide. 

a.  Page  2,  section  1.4,  Description  -  Font  differentiation 
to  indicate  features  that  are  not  currently  available:  Helvetica 
font  does  not  provide  adequate  differentiation  from  standard 
font.  Features  not  available  should  be  shadowed  or  highlighted 
to  ensure  easy  discrimination. 

b.  Page  6,  section  3.1,  Introduction:  It  is  not  necessary 
to  differentiate  the  Graphical  User  Interface  (GUI)  from  the  AWRP 
display.  The  user  considers  the  mouse,  not  the  GUI,  as  the 
primary  means  of  interacting  with  the  AWRP  display.  Therefore, 
the  GUI  tutorial  (i.e.,  "A  GUI  provides  the  mechanism  through 
which  the  user  accesses  the  functionality...")  is  not  necessary 
in  the  User's  Guide.  Explaining  the  difference  between  the  GUI 
and  the  display  may  cause  confusion  among  users . 


c.  Page  7,  figure  2,  On  versus  Off  State:  Figure  2 
provides  a  good  visual  example  of  a  button  in  an  "On"  versus 
"Off"  state. 

d.  Page  8,  figure  3,  A  Slider:  The  arrows  do  not  clearly 
indicate  which  features  of  the  slider  are  labeled.  While  the 
"trough"  is  labeled  on  the  diagram,  it  is  never  mentioned  in  the 
text.  When  discussing  the  "second  method  of  manipulation,"  use 
of  the  slider  bar  is  mentioned;  however,  the  slider  bar  is  not 
labeled  on  the  diagram. 
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e.  Page  9,  figure  4,  Text  Input  Windows:  The  text  in 
section  3.2.4  refers  to  the  "hollow  box";  however,  the  "hollow 
box"  is  not  labeled  in  the  diagram. 

f.  Page  9,  section  3.3.1,  Plan  View  Window:  Two- 
Dimensional  and  3 -Dimensional  products  are  discussed.  It  is 
necessary  to  clarify  which  three  dimensions  are  being  considered 
since  the  AWRP  has  4-Dimensional  products. 

g.  Page  9,  section  3.3.1,  Plan  View  Window:  The  user  is 
instructed  to  see  the  "Flight  Level"  section  for  further 
directions.  No  section  number  is  indicated  to  direct  the  user  to 
the  appropriate  section.  Furthermore,  the  user  is  told  that  the 
"Flight  Level"  section  is  below  section  3.3.1;  however,  this 
section  does  not  follow  section  3.3.1.  If  the  user  refers  to  the 
Table  of  Contents  to  locate  the  "Flight  Level"  section,  the 
section  is  listed  as  being  on  page  25;  yet,  it  actually  appears 
on  page  26. 

h.  Page  9,  section  3.3.2,  Vertical  Cross  Section  Window: 
The  user  is  referred  to  a  different  section  (e.g.,  see  "Flight 
Path"  section)  for  a  complete  explanation  of  the  Vertical  Cross 
Section.  This  section  should  contain  all  information  regarding 
the  Vertical  Cross  Section.  The  user  should  only  be  required  to 
refer  to  information  previously  discussed  in  the  document  and  not 
information  discussed  several  pages  ahead. 

i.  Page  9,  section  3.3.2,  Vertical  Cross  Section  Window: 
The  section  does  not  tell  the  user  the  function  of  the  Vertical 
Cross  Section  Window. 

j.  Page  9,  section  3.4,  Description  of  the  AWRP  Display 

GUI:  Since  the  user  does  not  differentiate  the  GUI  from  the 

display,  the  GUI  and  the  display  should  be  considered  as  one 
entity  in  this  document.  Refer  to  the  display  and  GUI  simply  as 
the  AWRP  Display. 

k.  Page  9,  section  3.4.2,  Weather  Product  Select  Buttons: 
The  first  part  of  the  section  states  that  only  one  product  can  be 
viewed  at  a  time.  However,  the  last  sentence  in  the  section 
states  that  "If  you  wish  to  view  more  than  one  weather  product  at 
a  time,  see  the  section  below  on  Weather  Overlays."  The  Weather 
Overlays  section  is  not  located  below  the  section  in  which  it  is 
referenced  and  does  not  appear  in  the  User's  Guide  until  page  24. 
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It  also  has  been  assigned  an  incorrect  page  number  in  the  Table 
of  Contents. 

l.  Page  10,  section  3.4.2,  Modes:  A  mouse  click  should  be 
defined  as  the  depression  and  release  of  the  mouse  button.  Many 
uses  of  the  mouse  on  the  AWRP  require  more  action  than  a  simple 
mouse  click.  In  some  instances,  the  user  is  required  to  click 
and  drag  to  create  lines  or  boxes  (e.g.,  zoom  and  pan) .  Caution 
should  be  used  referring  to  "mouse  clicks"  in  the  general  sense. 

m.  Page  11,  section  3. 4. 2.1,  Text  Mode:  The  text  mode  is 
-only  valid  for  some  products.  These  products  should  be  listed 

(e.g.,  AIRMETs,  SIGMETs,  and  Surface  Aviation  Observations 
[SAOs]  )  . 

n.  Page  12,  section  3. 4. 2.1,  Text  Mode:  The  last  part  of 
the  section  provides  a  good  explanation  of  how  to  obtain  text  for 
an  AIRMET  or  SIGMET. 

o.  Page  12,  section  3. 4. 2. 2,  Flight  Route  Select  Mode  : 
"Flight  Route"  and  "Flight  Path"  are  used  interchangeably  in  the 
document.  Convert  all  references  to  "Flight  Route."  The  "Clear 
Route"  and  "Show  Vertical"  buttons  are  discussed  in  the  section; 
however,  no  reference  is  made  regarding  the  location  of  these 
buttons  on  the  display. 

p.  Page  13,  section  3. 4. 2. 3,  Zoom  Mode:  Toward  the  end  of 
the  section,  the  user  is  told  that  using  the  middle  mouse  button 
zooms  in  a  specified  percentage.  The  percentage  needs  to  be 
defined  and  it  needs  to  be  indicated  whether  or  not  the  user  can 
control  this  percentage.  Additionally,  the  user  is  told  to  use 
the  "Function  Menu"  to  zoom  out  to  view  the  entire  continental 
United  States.  No  mention  is  made  regarding  the  location  of  the 
menu  on  the  display. 

q.  Page  13,  section  3. 4. 2. 4,  Pan  Mode:  Caution  should  be 
exercised  when  assuming  the  user  is  thinking  incorrectly  (e.g., 

"It  is  incorrect  to  think  of  this  feature  as  ...).  This  type  of 
wording  may  have  a  negative  impact  on  the  user. 

r.  Page  16,  section  3. 4. 2. 6,  Positional  Locator  Mode: 
Exiting  out  of  the  Positional  Locator  is  not  explained  in  the 
text.  Earlier  in  the  document  (see  section  3.4.2),  the  user  was 
told  to  choose  a  new  mode  to  exit  an  old  mode.  This  does  not 
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work  in  the  Positional  Locator  mode.  To  exit  from  the  Positional 
Locator  window,  the  user  must  activate  a  different  mode  and  then 
click  on  the  exit  button  in  the  Positional  Locator  window  to 
close  the  window.  However,  this  exit  button  is  "grayed  out"  when 
the  Positional  Locator  Mode  is  active  (see  figure  5) .  This  . 
leaves  the  user  with  no  way  to  close  the  Positional  Locator 
window  while  in  the  Positional  Locator  Mode.  This  is  not 
consistent  with  the  text  mode  where  the  user  can  close  the  text 
window  via  the  exit  button  while  still  in  the  text  mode. 

s.  Page  18,  section  3. 4. 2. 6,  Positional  Locator  Mode: 

When  using  symbology  not  considered  standard  knowledge  for  the 
user  (e.g..  Iron  Cross),  provide'  a  depiction  of  the  symbol. 

t.  Page  18,  section  3. 4. 2. 6,  Positional  Locator  Mode: 

When  reguiring  a  user  to  input  data  in  a  specific  format,  provide 
an  example  that  the  user  can  reference.  Always  specify  whether 
the  items  should  be  separated  by  forward  slash  or  backslash. 

Also,  be  consistent  in  discussing  the  data  entry  format.  In  one 
instance,  the  user  is  told  that  the  bearing  should  be  entered 
before  the  range;  however,  two  paragraphs  later,  the  user  is  told 
that  all  positions  will  be  returned  in  the  range/bearing  format. 

u.  Page  35,  section  4.3.1,  Icing  Potential:  Valid  time 
and  run  time  are  referred  to  in  the  text.  Some  users  may  not  be 
familiar  with  model  data;  therefore,  valid  time  and  run  time 
should  be  defined. 

V.  Page  44,  section  4.3.5,  table  1,  Flight  Category 
Definitions:  Users  do  not  need  to  know  Boolean  Logic  to 

understand  the  table;  therefore,  the  operators  column  labeled 
"boolean"  should  be  relabeled. 
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5.1.2  Operational  Analysis  Stage. 

This  section  will  present  the  results  from  the  operational 
analysis  portion  of  the  evaluation.  It  will  discuss  the 
comparison  of  the  AWRP  display  against  user  requests  from  the 
ATUWG  and  the  1993  Demval .  This  section  will  also  address  how 
well  the  AWR  products  could  be  expected  to  fulfill  user  needs  in 
an  operational  setting.  This  was  accomplished  by  comparing 
products  and  functions  to  controller  job  tasks  and  weather 
information  needs. 

5. 1.2.1  ATUWG  and  1993  Demval  Requests. 

Requests  from  the  ATUWG  and  the  1993  Demval  were  compared  to  the 
AWR  products,  functions,  and  interface  characteristics.  Results 
are  presented  in  tables  4,  5,  and  6.  User's  requests  were  based 
on  operational  needs.  While  some  requests  may  be  beyond  the 
scope  of  the  AWRP,  these  products  and  features  should  be  viewed 
as  required  to  successfully  provide  all  necessary  information  to 
specialists/controllers  in  an  operational  setting.  These 
requests  are  outlined  in  the  subsequent  paragraphs. 

5. 1.2. 1.1  ATUWG  General  User  Requests. 

Table  4  presents  the  results  for  the  ATUWG  general  user  requests . 
User  Requests  stated  in  table  4  not  met  by  the  AWRP  are  as 
follows: 

a.  User  Request  1:  A  busy  indicator  was  implemented; 
however,  it  does  not  appear  immediately  following  user  input.  It 
can  take  up  to  several  seconds  to  appear.  The  busy  indicator 
should  appear  immediately  following  input. 

b.  User  Request  2:  A  clear  button  to  return  the  system  to 
default  settings  has  not  been  implemented. 

c.  User  Request  4:  Users  requested  Pilot  Reports  (PIREPs) 
be  displayed  on  the  AWRP;  however,  PIREPs  are  not  displayed. 

PIREP  information  in  the  area  of  icing  and  turbulence  are  very 
important  to  controllers.  PIREPs  should  be  available  as  an 
overlay  product. 
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TABLE  4.  AIR  TRAFFIC  USER  WORKING  GROUP  MEETING  REQUIREMENTS 


USER  REQUEST 

MET? 

COMMENTS 

1 .  Implement  busy  icon  to 
indicate  when  the  system  is 
processing. 

No 

Indicator  exists,  but  it 
does  not  reflect  system 
status . 

2 .  Provide  clear  button  to 
return  to  default  settings. 

No 

Has  Clear  route  button. 

No  overall  clear  button. 

3.  Indicate  growth,  decay, 
and  intensity  for 
precipitation . 

Yes 

Indicates  storm  trend, 
area,  and  cell  motion. 

4 .  Provide  PIREP 
information. 

No 

5.  Provide  Instrument  Flight 
Rule  (IFR)  and  Mountain 
Obscuration  AIRMETs . 

Yes 

6.  Provide  forecast  Flight 
Category  product. 

No 

Currently,  only  SAO  data 
is  provided. 

7.  Make  lat/long  readouts 
active  only  in  the  FIX  mode. 

Yes 

The  product  is  very  slow. 

8.  Display  AGFS  time  with 
issue  and  current  time. 

Yes 

9.  Contour  winds  30  kts  or 
greater/display  winds  in 

DDSSS (D-Direction,  S-Speed) . 

Yes 

30  kt  winds  contoured. 

Wind  barbs  still  exist. 

Barbs  should  remain . 

10.  Add  dynamic  overlays  for 
downed  radars . 

Yes 

User ' s  manual  says 
feature  exists,  but  it 
has  not  been  observed 
since  system  started. 

11.  Overlays  for  jet 
airways . 

No 

Only  five  jet  airways 
exist  in  overlay. 
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TABLE  4 .  AIR  TRAFFIC  USER  WORKING  GROUP  MEETING  REQUIREMENTS 


(Continued) 


USER  REQUEST 

MET? 

COMMENTS 

12 .  Add  sustained  wind 

AIRMET . 

No 

Not  available.  : 

13 .  Alarm  for  SAO  updates 

No 

14.  Ability  to  scroll  SAO 
text  window. 

Yes 

Can  also  clear  text 
window. 
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TABLE  5.  AIR  TRAFFIC  USER  WORKING  GROUP  COLOR  REQUIREMENTS 


USER  REQUEST 

MET? 

COMMENTS 

1 .  Background  -  Black 

Yes 

2 .  Flight  Route  -  Orange 

No 

It  is  white.  Does  not 
contrast  well  with  Icing 
and  Mtn.  Obscuration 

AIRMETs . 

3.  Icing  -  Lt .  Blue 

No 

It  is  white.  Does  not 
contrast  well  with  mtn. 
obscuration  AIRMETs. 

4 .  Winds  -  White 

No 

Barbs  are  green.  They  do 
not  contrast  with  70  and 

130  kt  contours.  White 
would  provide  good 
contrast . 

5 .  Terrain  -  Brown 

Yes 

6.  Terrain  1500'  Rule  -  Tan 

No 

Current  color  is  Gray. 

While  gray  provides  good 
contrast,  users  should  be 
polled. 

7 .  Temperature  -  Red 

Yes 

8 .  Storm  Movement  -  Teal 

Yes 

9 .  Area  Movement  -  White 

Yes 

10.  State  Borders  -  Blue 

Yes 

11.  LIFR  -  Orange 

No 

It  is  red.  Should  be 
orange . 

12.  IFR  -  Red 

No 

It  is  yellow.  Should  be 
red,  IFR  is  no-go  for 

AFSS . 

13.  Marginal  Visual  Flight 

Rules  (MVFR)  -  Lt .  Blue 

No 

Blue  is  okay  for  TMU. 
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TABLE  5.  AIR  TRAFFIC  USER  WORKING  GROUP  COLOR  REQUIREMENTS 

(Continued) 


USER  REQUEST 

MET? 

COMMENTS 

14.  Visual  Flight  Rules 
(VFR)  -  Green 

Yes 

15.  Turbulence  -  Yellow 

Yes 

16.,  Lightning- 
White  (Imin) /Gray (4min) 

No 

Yellow  is  5  minutes  old. 

White  is  1  minute  old. 
Neither  contrast  well 
against  Level  3  and  4 
precipitation. 

17.  Convective  SIGMETs  - 
Fuscia 

Yes 

18.  Mtn.  Obscuration  -  White 

No 

Gray  currently.  Does  not 
provide  contrast  with 
current  icing  color. 

19,  AIRMETs  -  solid  lines 

Yes 

20.  SIGMETs  -  dashed  lines 

No 

Convective  and 

Miscellaneous  SIGMETs  are 

solid  lines. 

21.  VORs  -  White 

Yes 

22.  Airports  -  Yellow 

Yes 

i 

23 .  Center/Sector  Boundaries 
-  Green 

Yes 
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TABLE  6.  1993  DEMVAL  REQUIREMENTS 


USER  REQUEST 

MET? 

1 .  Display  wind  in  numeric 
format  since  barbs  are 
difficult  to  read. 

No 

2 .  Incorporate  lightning 
playback  or  history  loop. 

No 

3 .  Be  able  to  display  cross 
section  using  pre-selected 
range  of  altitude. 

No 

4 .  Incorporate  12  and  24 
hour  forecasts. 

No 

5.  Add  all  public  use 
airports  to  airport  overlay. 

No 

6 .  Incorporate  use  of 
function  keys. 

No 

7.  Add  satellite  imagery, 
looping,  and  PIREPs. 

No 

8.  Incorporate  a  continuous 
location  readout. 

No 

9 .  Add  storm  area  movement 
to  the  Radar  Mosaic . 

Yes 

10.  Increase  range  of  wind 
contours  down  to  30  kts. 

Yes 

11 .  Incorporate  approach 
control  boundaries  to  low 
sector  overlay. 

No 

12 .  Add  capability  to 
overlay  SIGMETs  on  the  Radar 
Mosaic. 

— — — — — _ L 

Yes 

COMMENTS 


Note :  Should  not  change , 

Human  Factors  standards 
suggest  barbs  since  they 
indicate  wind  direction. 


d.  User  Request  6:  In  addition  to  plotting  SAG  reports, 
users  have  expressed  a  desire  to  have  Terminal  Forecast  (FT) 
reports  plotted  as  well.  In  essence,  this  would  provide  a 
forecasted  Flight  Category  product.  This  has  not  been 
implemented  on  the  AWRP  display. 

e.  User  Request  11:  The  current  AWRP  jet  overlay  database 
only  contains  five  jet  airways.  All  jet  airways  should  be 
available  to  the  user.  This  database  should  be  updated. 

f .  User  Request  12 :  Users  requested  sustained  wind 
AIRMETs;  however,  these  AIRMETs  have  not  been  implemented  on  the 
display. 

g.  User  Request  13:  While  SAG  reports  are  displayed, 
users  requested  an  audible  indication  when  SAG  reports  were 
updated.  No  such  indication  exists  on  the  current  AWRP  display. 

5. 1.2. 1.2  ATUWG  Color-Coding  Recmests. 

Table  5  presents  the  comparison  results  from  the  ATUWG 
product/display  color-coding  requirements.  User  requests  stated 
in  table  5  not  met  by  the  AWRP  are  as  follows: 

a.  User  Request  2 :  Users  requested  an  orange  flight 
route.  It  is  currently  white.  This  may  only  present  a  problem 
if  the  user  is  drawing  a  route  in  the  Icing  Product  with  AIRMETs 
overlaid.  In  this  case,  the  route  may  not  be  distinguishable 
against  the  Icing  AIRMET. 

b .  User  Request  3 :  Users  requested  light  blue  Icing 
AIRMETs;  however,  the  current  Icing  AIRMET  is  outlined  in  white. 
While  a  light  blue  outline  of  the  AIRMET  may  not  contrast  with  a 
light  blue  gridded  icing  field,  the  current  white  AIRMET  does  not 
contrast  with  the  gray  mountain  obscuration  AIRMET.  Better 
contrast  needs  to  be  provided. 

c.  User  Request  4:  White  wind  barbs  were  requested  by 
users.  Wind  barbs  on  the  current  AWRP  display  are  light  green. 
The  light  green  wind  barbs  are  difficult  to  distinguish  in  the 
70-  and  130-kt  wind  contours  as  these  contours  are  lighter  shades 
of  green.  White  wind  barbs  would  provide  better  contrast  and 
easy  discrimination  against  all  wind  contour  colors. 
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d.  User  Requests  11-14 :  The  current  Flight  Category  color 
scheme  is  as  follows: 

1.  Low  Instrument  Flight  Rule  (LIFR)  -  red, 

2  .  IFR  -  yellow, 

3 .  MVFR  -  blue ,  and 

4 .  VFR  -  green . 

AFSS  and  ARTCC  users  had  requested  the  following: 

1.  LIFR  -  orange, 

2.  IFR  -  red, 

3.  MVFR  -  light  blue,  and 

4 .  VFR  -  green . 

NOTE :  Since  AFSS  users  and  ARTCC  users  experience  hazardous 

conditions  under  different  categories,  color-coding 
should  be  different  for  each  of  these  user  groups.  For 
AFSS  users,  critical  decisions  occur  during  IFR 
conditions;  therefore,  the  color  scheme  for  AFSS  users 
should  be : 

1.  LIFR  -  orange, 

2.  IFR  -  red, 

3 .  MVFR  -  yellow,  and 

4.  VFR  -  green. 

Since  critical  decisions  for  ARTCC  users  begin  during 
LIFR  conditions,  the  current  color  scheme  is 
appropriate . 

e.  User  Request  16:  Currently,  1-minute-old  lightning  is 
white  and  2  to  5-minute-old  lightning  is  yellow.  Neither  the 
white  nor  yellow  strikes  contrast  well  with  the  reflectivity 
levels  on  the  Radar  Mosaic.  Lightning  strikes  should  be  easy  to 
differentiate  when  overlaid  on  the  Radar  Mosaic. 

f.  User  Request  18:  The  current  gray  Mountain  Obscuration 
AIRMETs  are  difficult  to  differentiate  from  the  white  Icing 
AIRMETs.  Better  contrast  should  be  provided. 
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g.  User  Request  20:  SIGMETs  should  be  drawn  using  broken 
or  dashed  lines.  Convective  SIGMETs  and  miscellaneous  SIGMETs 
are  drawn  using  solid  lines.  These  lines  should  be  dashed  to 
remain  consistent  with  all  other  SIGMETs  (i.e.,  icing  and 
turbulence) . 

5. 1.2. 1.3  1993  Demval  ■Recme.g-t.s. . 

Table  6  presents  the  comparison  results  for  user  requests  made 
during  the  1993  Demval .  The  AWRP  display  did  not  meet  all  user 
requests  as  outlined  in  table  6.  Results  were  as  follows: 

a.  User  Request  1:  Users  had  requested  Winds  Aloft  be 
displayed  in  an  all  numeric  presentation  (e.g.,  DDDSS) . 

NOTE:  The  current  staff  and  barb  depiction  allows  for 

easy  pattern  perception  of  overall  wind  flow 
patterns.  The  current  depiction  should  remain 
as  it  follows  human  factors  design  principles. 

b.  User  Request  2:  A  lightning  playback  or  looping 
feature  was  requested.  This  feature  has  not  been  implemented. 

The  need  for  a  looping  feature  was  discussed  in  section 

5 . 1 . 1 . 1 . 1 ,  item  d. 

c.  User  Request  3:  Users  indicated  they  wanted  to  only 
see  the  cross  section  window  for  a  preselected  range  of  altitudes 
(i.e.,  4000-10000  feet).  Currently,  the  cross  section  displays 
all  vertical  levels.  AFSS  Specialists  indicated  that  their 
primary  concern  is  weather  2  to  3  flight  levels  above  and  below 
the  route  of  flight. 

d.  User  Request  4:  Twelve  and  24 -hour  forecasts  were 
requested.  AWRP  currently  provides  hourly  forecasts  out  to  6 
hours.  Twelve  and  24 -hour  forecasts  are  often  used  by 
specialists  for  pilot  weather  briefings  and  outlook  briefings. 

e.  User  Request  5:  AFSS  Specialists  would  require  public 
use  airports  on  the  AWRP  overlay.  These  airports  are  not 
currently  on  the  overlays.  Additionally,  the  current  Victor 
Airway  database  is  incomplete.  All  overlay  databases  need  to  be 
completed  and  updated. 
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f.  User  Request  7:  AFSS  Specialists  utilize  satellite 
imagery  on  a  daily  basis  and  have  requested  it  as  a  product  on 
the  AWRP  display.  Satellite  imagery  should  be  incorporated  into 
the  display. 

g.  User  Request  11:  Users  requested  the  addition  of 
approach  control  boundaries  to  the  low  sector  overlay.  They  have 
stated  this  overlay  would  be  useful  in  determining  if  weather  was 
impacting  the  terminal  airspace. 

5 . 1 . 2 . 2  JTA  Comparison . 

Products  and  features  on  the  AWRP  were  compared  to  the  weather 
information  requirements  of  Inflight  Specialists,  Preflight 
Specialists,  Enroute  Flight  Advisory  Specialists  (EFAS) ,  and 
ARTCC  TMCs  as  defined  in  ACT-320 's  JTA  report.  The  comparison 
considered  what  information  was  needed  and  how  that  information 
was  presented. 

5. 1.2. 2.1  Information  Requirements. 

While  this  evaluation  considers  products  for  their  individual 
merits,  the  collection  of  AWR  products  was  also  assessed  for  its 
completeness  in  providing  users  with  all  required  weather 
information.  This  section  will  examine  the  extent  to  which  the 
AWRP  meets  the  information  needs  for  the  positions  listed  in 
paragraph  5, 1.2. 2.  Information  requirements,  as  determined  in 
the  JTA,  were  compared  to  information  provided  by  products  and 
functions  on  the  AWRP  display.  Requirements  are  listed  by 
position,  but  are  not  linked  to  specific  job  tasks  associated 
with  that  position.  For  a  complete  analysis  of  information 
requirements  for  each  position's  job  tasks,  see  the  JTA  diagrams 
and  tables  listed  in  appendix  A. 

.5_.  1.2. 2. 1.1  AFSS  Preflight  Specialists. 

The  primary  weather-related  job  task  of  a  preflight  specialist  is 
to  provide  preflight  weather  briefings  to  General  Aviation  (GA) 
pilots.  In  order  to  accomplish  this  task,  the  specialist  is 
required  to  have  knowledge  of  and  provide  several  types  of 
weather  information  to  pilots.  Table  7  lists  a  preflight 
specialist ' s  weather  information  needs  as  defined  by  the  JTA  and 
indicates  whether  they  were  satisfied  by  the  AWR  products.  As 
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noted  in  table  7,  the  AWRP  provides  preflight  specialists  with  7 
complete  products  and  2  partial  products  but  of  15  total 
products  needed  to  complete  their  weather-related  job  tasks. 
Additionally,  in  providing  an  Outlook  Briefing  (based  on  weather 
conditions  expected  beyond  6  hours) specialists  would  not  be  able 
to  retrieve  the  necessary  infoirmation  from  the  AWRP.  The 
comparison  results  suggest  that  if  the  AWRP  was  currently  being 
used  by  preflight  specialists  in  an  operational  setting,  the 
display  would  have  to  be  supplemented  with  additional  weather 
systems  (e.g..  Model  1  Full  Capacity  [MIFC]  or  vendor  weather 
systems) . 

5.1.2.2.1.2  AFS5  Inflight  Specialists. 

The  primary  weather-related  job  tasks  of  an  inflight  specialist 
are  to  provide  weather  briefings,  issue  broadcasts,  and  issue 
local  airport  advisories  to  GA  pilots.  In  order  to  accomplish 
these  tasks,  the  specialist  is  required  to  have  knowledge  of  and 
provide  several  types  of  weather  information.  Table  8  lists  an 
inflight  specialist's  weather  information  needs  as  defined  by  the 
JTA  and  indicates  whether  they  were  satisfied  by  the  AWR 
products . 

As  shown  in  table  8,  the  AWRP  provides  inflight  specialists  with 
8  complete  products  and  2  partial  products  out  of  18  total 
products  needed  to  complete  their  weather-related  job  tasks. 
Similar  to  the  preflight  specialist,  inflight  specialists  issue 
Outlook  Briefings  requiring  forecasts  beyond  6  hours.  Again, 
results  suggested  that  supplemental  weather  systems  would  be 
necessary  in  an  operational  setting. 
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TABLE  7.  PREFLIGHT  SPECIALIST'S  WEATHER  INFORMATION  REQUIREMENTS 


WEATHER  REQUIREMENT 

AVAILABLE  ON  AWRP? 

Radar  Mosaic 

Yes 

LLWAS 

No 

Convective  Activity 
(including  lightning) 

Yes 

Icing 

Yes 

Turbulence 

Yes 

Weather  Advisories 
(AIRMETs,  SIGMETs,  Center 
Weather  Advisories  [CWAs] , 
Severe  Weather  Watch) 

AIRMETs,  SIGMETS  -  Yes 

CWAs,  Severe  Weather  Watch  - 
No 

Synopsis  (frontal 
activity,  pressure  areas) 

No 

Terminal  Forecasts 

No 

Current  Conditions  (SAO, 
PIREPs) 

SAOs  -  Yes 

PIREPs  -  No 

Satellite  Imagery 

No 

Area  Forecast 

No 

Prognosis  Charts  (12-24hr) 

No 

Winds  Aloft 

Yes 

Upper  Air  Moisture 

No 

Weather  Depiction  (VFR, 

IFR  conditions) 

Yes 
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TABLE  8.  INFLIGHT  SPECIALIST'S  WEATHER  INFORMATION  REQUIREMENTS 


WEATHER  REQUIREMENT 

AVAILABLE  ON  AWRP? 

Radar  Mosaic 

Yes 

LLWAS 

No 

Convective  Activity 
(including  lightning) 

Yes 

Icing 

Yes 

Turbulence 

Yes 

Weather  Advisories  (AIRMETs, 
SIGMETs,  CWAs,  Severe 

Weather  Watch) 

AIRMETs,  SIGMETs  -  Yes 

CWAs,  Severe  Weather  Watch  - 
No 

Synopsis  (frontal  activity, 
pressure  areas) 

No 

Terminal  Forecasts 

No 

Current  Conditions  (SAO, 

PIREPs) 

SAOs  -  Yes 

PIREPs  -  No 

Satellite  Imagery 

No 

Area  Forecast 

No 

Prognosis  Charts  (12-24hr) 

No 

Winds  Aloft 

Yes 

Upper  Air  Moisture 

No 

Runway  Visual  Range  (Airport 
dependent) 

No 

Airport  Wind  Direction  and 
Speed 

Yes,  via  the  textual  SAO 
reports 

Wake  Turbulence 

No 

Weather  Depiction  (VFR,  IFR 
conditions) 

Yes 
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5 , 1 , 2 , 2  T 1 • 3 _ AFSS  EFAS  Specialists. 


The  primary  weather-related  job  tasks  of  an  EFAS  specialist  are 
to  disseminate  enroute  weather,  interpret  unforecasted  weather 
trends,  and  solicit  PIREPs .  In  order  to  accomplish  these  tasks, 
the  specialist  is  required  to  have  knowledge  of  and  provide 
several  types  of  weather  information.  Table  9  lists  an  EFAS 
specialist's  weather  information  needs  as  defined  by  the  JTA  and 
indicates  whether  they  were  satisfied  by  the  AWR  products. 

As  noted  in  table  9,  the  AWRP  provides  EFAS  specialists  with  8 
complete  products  and  2  partial  products  out  of  18  total  products 
needed  to  complete  their  weather-related  job  tasks.  The  AWRP 
would  require  supplemental  weather  systems  to  provide  appropriate 
weather  information  for  completing  weather-related  job  tasks  in 
an  operational  environment . 

5^.1 . 2 . 2  ^1^._.ARTCC  Traffic  Management  Coordinators. 

The  primary  job  task  of  an  ARTCC  TMC  is  to  provide  a  safe  and 
efficient  traffic  flow  into,  out  of,  and  within  the  ARTCC 's 
airspace.  While  a  TMC  is  not  required  to  perform  weather 
specific  job  tasks,  weather  is  a  crucial  factor  in  managing  an 
ARTCC's  traffic  flow.  In  providing  this  traffic  management 
function,  the  TMC  will  often  reroute  and  meter  traffic  to  safely 
divert  airplanes  around  hazardous  weather.  In  order  to 
accomplish  this  task,  the  TMC  is  required  to  have  knowledge  of 
several  types  of  weather  information.  Table  10  lists  a  TMC's 
weather  information  needs  as  defined  by  the  JTA  and  indicates 
whether  they  were  satisfied  by  the  AWR  products. 

As  noted  in  table  10,  the  AWRP  provides  TMCs  with  5  complete 
products  and  3  partial  products  out  of  13  total  products  needed 
to  complete  their  job  tasks.  The  AWRP  would  require  supplemental 
weather  systems  to  provide  a  TMC  with  the  appropriate  weather 
information  to  complete  job  tasks  in  an  operational  environment. 
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TABLE  9.  EFAS  SPECIALIST'S  WEATHER  INFORMATION  REQUIREMENTS 


WEATHER  REQUIREMENT 

AVAILABLE  ON  AWRP? 

Radar  Mosaic 

Yes 

LLWAS 

No 

Convective  Activity 
(including  lightning) 

Yes 

Icing 

Yes 

Turbulence 

Yes 

Weather  Advisories 
(AiRMETs,  SIGMETs,  CWAs, 
Severe  Weather  Watch) 

AiRMETs,  SIGMETs  -  Yes 

CWAs,  Severe  Weather  Watch  - 
No 

Synopsis  (frontal 
activity,  pressure  areas) 

No 

Terminal  Forecasts 

No 

Current  Conditions  (SAO, 
PIREPs) 

SAOs  -  Yes 

PIREPs  -  No 

Satellite  Imagery 

No 

Area  Forecast 

No 

Prognosis  Charts  (12-24hr) 

No 

Winds  Aloft 

Yes 

Upper  Air  Moisture 

No 

Runway  Visual  Range 
(Airport  dependent) 

No 

Airport  Wind  Direction  and 
Speed 

Yes 

Wake  Turbulence 

No 

Weather  Depiction  (VFR, 

IFR  conditions) 

Yes 
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TABLE  10.  ARTCC  TMC's  WEATHER  INFORMATION  REQUIREMENTS 


WEATHER  REQUIREMENT 

AVAILABLE  ON  AWRP? 

Radar  Mosaic 

Yes 

Convective  Activity 
(including  lightning) 

Yes 

Icing 

Yes 

Turbulence 

Yes 

Weather  Advisories 

AIRMETs,  SIGMETs  -  Yes 

(AIRMETs,  SIGMETs,  CWAs, 

CWAs,  Severe  Weather  Watch  - 

Severe  Weather  Watch) 

No 

Terminal  Forecasts 

No 

Current  Conditions  (SAO, 

SAOs  -  Yes 

PIREPs) 

PIREPs  -  No 

Satellite  Imagery 

No 

Winds  Aloft 

Yes 

Jet  Stream 

Partially  (barbs  provide 
direction) 

Volcanic  Ash 

No 

Surface  Wind 

No 

Mountain  Wave 

No 
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5. 1.2. 2. 2  Information  Presentation. 

This  section  will  examine  the  extent  to  which  the  AWRP  overcomes 
the  information  presentation  problems  of  current  weather  systems 
as  outlined  in  the  JTA.  Information  presentation  problems  can 
be  classified  as  problems  with  data  display,  data  entry,  or  data 
retrieval.  .  , 

Several  issues  associated  with  the  presentation  of  information 
were  identified  in  the  JTA.  These  issues  were  (all  references  in 
this  paragraph  refer  to  the  Job  Task  Analysis  report  completed  by 
ACT-320) : 

a.  Accessing  and  comparing  weather  information  from 
several  systems  to  ensure  weather  accuracy  (see  paragraph 

5. 2. 2.1,  5. 2. 3. 2)  is  time  consuming  and  may  lead  to  increased 
task  difficulty  as  well  as  increased  levels  of  workload. 

b.  Attempting  to  relate  graphical  and  textual  weather 
information  (see  paragraph  5. 2. 2.1)  may  increase  the  amount  of 
mental  interpretation  required  to  properly  utilize  products. 
Increased  interpretation  may  lead  to  increased  levels  of  task 
difficulty  and  workload. 

c.  The  need  to  read,  interpret,  and  mentally  filter 
several  pages  of  MIFC  text  (see  paragraph  5. 2. 2.1)  is  time 
consuming  and  inefficient. 

d.  Locally  produced  graphics  are  an  attempt  at  reducing 
the  amount  of  textual  information  (see  paragraph  5. 2.1.1 ); 
however,  the  process  of  creating  these  graphics  is  time  consuming 
and  needs  to  be  improved . 

e.  Command  line  interfaces  (typing  in  requests)  on  some  of 
the  current  weather  systems  are  tedious  and  time  consuming  (see 
paragraph  5.2. 3.1).  Additionally,  this  interaction  becomes  time 
consuming  and  frustrating  when  typographical  errors  are  made. 

f.  Current  vendor  graphic  weather  systems  are  based  on  a 
user  interface  that  is  not  efficient  and  requires  too  many  user 
inputs  for  each  request  and/or  action  (see  paragraph  5. 2. 3.1). 
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g.  Mentally  placing  PIREPs  along  a  route  of  flight  can  be 
tedious  and  time  consuming  (paragraph  5. 2. 2.1). 

Given  the  issues  identified  in  the  JTA,  the  AWRP  may  provide  the 
following  improvements: 

a.  Issue  A:  Until  the  AWRP  provides  all  required  weather 
information,  users  would  still  need  to  access  other  systems. 

b.  Issue  B:  Since  the  APJRP  does  not  provide  all  necessary 
weather  information,  users  would  need  to  retrieve  additional 
information  from  other  sources.  Most  of  this  information  would 
be  presented  in  a  textual  format.  Thus,  the  comparison  of 
textual  and  graphical  information  would  still  be  required. 

c.  Issue  C:  The  AWRP  begins  to  alleviate  this  problem  as 
it  presents  more  information  graphically. 

d.  Issue  D:  The  AWRP  overcomes  the  manual  plotting 
process  as  the  AWRP  system  provides  advisories  in  a  graphical 
format.  This  feature  is  not  unique  to  the  AWRP,  however,  since 
software  can  be  purchased  or  created  that  automatically  plots 
this  information. 

e.  Issue  E:  Since  the  AWRP's  primary  input  device  is  a 
mouse,  the  system  does  overcome  the  problem  of  typing  errors. 
However,  in  many  instances  (e.g.,  changing  altitude  and 
retrieving  SAOs) ,  the  AWRP  does  not  have  an  acceptable  response 
speed. 

f.  Issue  F:  While  the  current  menu  structure  on  the  AWRP 
represents  an  improvement  as  it  is  better  organized  and  decreases 
user  memory  reliance,  the  AWRP  menu  structure  still  needs  to  be 
improved  further. 

g.  Issue  G:  The  AWRP  does  not  offer  any  relief  from  the 
PIREP  problem  as  PIREPs  are  not  available . 
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While  the  AWRP  was  not  developed  to  satisfy  the  JTA,  the  AWRP 
does  not  address  two  of  the  more  important  conclusions  from  the 
JTA.  First,  it  was  suggested  that  products  and  functions  be 
tailored  to  meet  the  needs  of  user's  job  tasks.  The  AWRP  was 
designed  to  meet  the  needs  of  both  AFSS  Specialists  and  ARTCC 
TMCs.  Since  the  requirements  of  these  positions  are  inherently 
different,  products  do  not  fulfill  the  needs  of  either  group. 

Secondly,  it  was  suggested  that  products  be  integrated  on  either 
a  single  current  or  future  platfonn.  Currently,  no  operational 
platform  is  capable  of  receiving  the  AWRP.  Additionally,  given 
that  the  AWRP  does  not  provide  users  with  the  necessary  weather 
information  requirements,  the  AWRP  could  not  be  implemented  as  a 
single  weather  platform. 

5.2  METEOROLOGICAL  RESULTS  AND  DISCUSSION. 

5.2.1  Meteorological  Interpretation. 

In  the  following  paragraphs,  each  product  is  discussed  with 
regard  to  the  amount  of  meteorological  interpretation  needed. 

The  level  of  interpretation  for  each  product  was  assigned  a 
numerical  rating  from  1  to  10  with  1  being  no  interpretation 
needed  and  10  being  complete  interpretation  needed.  Table  11 
summarizes  the  numerical  interpretation  ratings  of  each  product. 


TABLE  11.  METEOROLOGICAL  INTERPRETATION  OF  AWRP 


Product 

Level  of 
Interpretation 

Radar  Mosaic 

2 

Storm  Attributes 

9 

Winds  Aloft 

2 

Temperatures  Aloft 

6 

AIRMETS  and  SIGMETS 

8 

Current  Flight  Category 

7 

In-Flight  Icing  Potential 

8 

Route  Winds 

2 
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5. 2. 1.1  Radar  Mosaic  and  Storm  Attributes. 


The  AWRP  Radar  Mosaic  displays  six  contour  levels  of  radar 
reflectivity.  The  higher  reflectivity  levels  correspond  to 
heavier  precipitation.  The  storm  attributes  consist  of 
individual  Storm  Cell  Motion  and  Area  Motion  through  the  use  of 
vectors  pointing  in  the  direction  of  movement.  The  storm 
attributes  also  include  cloud- to -ground  lightning  strikes 
displayed  and  updated  every  minute.  In  addition.  Convective 
SIGMETs  issued  by  the  Aviation  Weather  Center  (AWC)  can  overlay 
the  Radar  Mosaic.  The  level  of  interpretation  by  air  traffic 
personnel  of  the  Radar  Mosaic  should  be  minimal  because  the 
display  is  very  similar  to  current  operational  radar  mosaics. 
However,  the  storm  attributes  present  the  following 
interpretation  issues: 

a.  The  user  must  decide  which  motion  product  to  use.  The 
Cell  and  Area  Motions  are  automated  products  that  are  calculated 
every  5  minutes.  A  separate  area  motion  is  also  given  in  the 
Convective  SIGMET  as  determined  by  an  AWC  forecaster  viewing 
successive  radar  images.  Situations  were  observed  with  the  AWRP 
where  the  area  motion  is  in  one  direction,  the  cell  motion  is  in 
another  direction,  and  the  motion  given  in  the  Convective  SIGMET 
is  in  a  third  direction.  The  differences  in  area  and  cell  motion 
are  to  be  expected  since  the  area  motion  is  governed  by  the 
growth  and  decay  of  cells  within  an  area,  while  the  cells  are 
separate  entities  that  move  within  the  overall  area.  It  is  not 
apparent  to  the  user  when  Cell  Motion  should  be  used  rather  than 
Area  Motion  and  vice  versa.  One  way  of  addressing  this  issue 
would  be  for  Cell  Motion  to  be  available  only  on  the  terminal 
scale  (i.e.,  on  the  order  of  tens  of  kilometers  (km)). 

b.  If  both  the  Area  Motion  and  the  Convective  SIGMET  are 
to  be  displayed,  then  the  motions  in  each  should  be  in  general 
agreement.  While  differences  are  to  be  expected  since  the  Area 
Motion  is  updated  every  5  minutes  while  Convective  SIGMETs  are 
issued  once  per  hour,  the  presentation  of  two  different  area 
motions  is  confusing  and  potentially  dangerous.  An  example  is 
shown  graphically  in  figure  6.  The  area  motion  for  the  storms  in 
southeastern  Wisconsin  and  northeastern  Illinois  is  given  in 
Convective  SIGMET  25C  as  from  330°  (i.e.,  from  the  northwest)  at 
25  kts.  The  AWRP  Area  Motion  is  given  by  the  white  arrow  near 
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Milwaukee,  Wisconsin  (MKE) .  The  arrow  clearly  points  towards, 
not  from,  the  northwest  with  a  speed  of  movement  (shown  at  the 
head  of  the  arrow)  of  20  kts.  Thus,  the  AWRP  Area  Motion  and  the 
motion  given  in  the  Convective  SIGMET  are  completely  opposite. 

The  example  in  figure  6  was  not  an  isolated  occurrence. 

c.  A  third  issue  with  the  storm  attributes  concerns  the 
changing  outline  of  storm  areas  and  area  motion  defined  by  the 
AWRP.  With  5 -minute  updates,  the  area  outlines  and  the  motion 
vectors  fluctuate  considerably.  As  a  result,  significantly 
different  information  could  be  given  to  end  users  on  requests 
only  minutes  apart.  It  was  noted  that  storm  area  outlines  often 
would  split,  merge,  disappear,  change  shape  drastically,  or  any 
combination  of  the  above  from  one  5-minute  update  to  the  next. 
More  consistency  is  needed  in  the  products.  Some  fluctuation  is 
to  be  expected  and  occurs  naturally  within  storm  systems,  but  for 
enroute  storm  tracking,  the  overall  trend  of  the  system  is 
important.  The  Radar  Mosaic  does  not  require  a  high  level  of 
interpretation  and  is  assigned  a  numerical  rating  of  2.  However, 
too  much  and  potentially  conflicting  motion  information  is 
presented  in  the  storm  attributes.  Therefore,  the  storm 
attributes  product  is  ranked  separately  from  the  mosaic.  Since  a 
high  level  of  interpretation  is  needed  to  discern  which  motion 
information  to  use,  a  numerical  rating  of  9  is  assigned. 

5. 2. 1.2  Winds  Aloft. 

Winds  Aloft  displays  gridded  wind  information  by  using  a  staff 
pointed  in  the  direction  the  winds  are  coming  from  and  barbs  on 
the  staff  indicating  the  wind  speed.  To  aid  in  identifying  the 
specific  direction  the  wind  is  coming  from,  the  compass  direction 
in  tens  of  degrees  is  displayed  at  the  base  of  the  staff.  Filled 
contours  of  wind  speed  give  immediate  identification  of  the 
region  of  strongest  winds  at  a  flight  level.  The  AWRP  Winds 
Aloft  are  similar  to  current  operational  products.  Meteorological 
symbolism  is  standard  for  wind  depiction.  Little  to  no 
interpretation  should  be  needed  by  air  traffic  personnel. 
Therefore,  the  Winds  Aloft  level  of  interpretation  is  assigned  a 
numerical  rating  of  2 . 
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FIGURE  6.  AWRP  AREA  MOTION  AND  CONVECTIVE  SIGMET  MOTION 


5. 2. 1.3  Temperatures  Aloft. 

The  Temperatures  Aloft  product  displays  temperature  contours  at 
10°  Celsius  intervals  with  5°  intervals  next  to  the  0°  contour. 

At  10°  Celsius  intervals,  the  spacing  between  contours  can  be 
large  and  considerable  interpolation  between  contours  is  needed 
in  order  to  determine  the  temperature  at  a  specific  location.  If 
the  temperature  is  desired  along  a  route  of  flight,  the  use  of 
the  vertical  cross  section  yields  a  better  picture  that  requires 
less  interpolation.  Since  the  information  presented  in  the 
horizontal  plan  view  is  sparse,  the  Temperatures  Aloft  product 
level  of  interpretation  is  assigned  a  numerical  rating  of  6 . 

5 .2. 1.4  AIRMETs  and  SIGMETs. 

AIRMETs  and  SIGMETs  are  graphically  displayed  on  the  AWRP 
horizontal  plan  view.  Area  boundaries  are  shown  and  are  color 
coded  in  order  to  delineate  different  types  of  AIRMETs  and 
SIGMETs.  Updates  are  automatically  incorporated  by  the  AWRP  and 
displayed  as  the  AIRMETs  and  SIGMETs  become  available.  The 
graphical  display  of  AIRMETs  and  SIGMETs  requires  little 
interpretation  as  far  as  areal  extent  is  concerned.  However, 
other  than  the  color  coding  to  indicate  the  phenomena  of 
interest,  no  other  information  is  given.  For  example,  the  text 
for  an  Icing  AIRMET  must  be  retrieved  and  read  to  determine  the 
location  of  icing  (e.g.,  in  cloud  or  in  precipitation),  the  type, 
the  severity,  and  the  vertical  extent  of  the  Icing  Potential. 
Standard  meteorological  symbology  for  icing  types  and  the  base 
and  top  of  the  area  should  have  been  incorporated  into  the 
display.  Due  to  the  lack  of  graphical  information,  the  AIRMETs 
and  SIGMETs  level  of  interpretation  is  assigned  a  numerical 
rating  of  8 . 

5. 2. 1.5  Current  Flight  Category. 

The  Current  Flight  Category  Product  displays  a  color  filled 
circle  at  surface  weather  observation  locations  throughout  the 
United  States.  Different  colors  correspond  to  the  different 
flight  categories  of  Visual  Flight  Rules  (VFR) ,  Marginal  Visual 
Flight  Rules  (MVFR) ,  Instrument  Flight  Rules  (IFR) ,  Low 
Instrument  Flight  Rules  (LIFR) .  When  an  obstruction  to 
visibility  occurs  at  a  location,  the  weather  symbol  for  the 
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obstruction  is  plotted  next  to  the  location.  AWRP  weather 
symbols  are  standard  meteorological  symbols  that  air  traffic 
personnel  are  required  to  know.  The  level  of  interpretation  for 
individual  airport  locations  is  minimal.  However,  not  enough 
information  is  given  in  the  following  two  areas: 

a.  There  is  no  information  on  areas  between  airports.  The 
existence  of  conditions  between  airports  is  important  from  a 
general  aviation  standpoint.  Since  no  information  is  given, 
considerable  interpretation  will  be  done  for  these  areas.  For 
example,  accepted  meteorological  practices  call  for  contouring  of 
IFR  and  VFR  areas  based  upon  reports  from  airport  locations  and 
knowledge  of  terrain  (e.g.,  valleys  and  ridges).  Even  though 
technically  it  cannot  be  stated  that  regions  will  be  VFR  between 
two  known  locations  of  VFR  conditions,  the  tendency  will  be  to 
form  contours  enclosing  the  two  locations  and  the  between 
regions.  As  a  result,  it  is  expected  that  considerable  mental 
contouring  (i.e.,  interpretation)  of  the  AWRP  display  will  take 
place . 

b.  There  are  no  forecasted  flight  conditions.  Thus,  there 
is  no  information  available  for  flight  planning  purposes.  Users 
will  be  forced  to  obtain  additional  text  information,  such  as 
terminal  forecasts,  to  interpolate  into  the  future. 

Due  to  the  presence  of  large  regions  without  information  and  the 
lack  of  forecast  information,  the  current  flight  category  product 
requires  a  high  level  of  interpretation  and  is  assigned  a 
numerical  rating  of  7 . 

5. 2. 1.6  Inflight  Icing  Potential. 

The  Inflight  Icing  Potential  combines  Icing  AIRMETs  with  gridded 
areas  of  automated  Icing  Potential.  The  Icing  AIRMETs  are 
displayed  at  all  flight  levels,  however,  the  automated  Icing 
Potential  is  only  displayed  if  icing  potential  is  identified  at 
the  particular  flight  level  of  interest.  As  noted  in  paragraph 
5. 2. 1.4,  the  graphical  Icing  AIRMET  does  not  display  any 
information  on  the  type  or  severity  of  icing.  The  automated 
Icing  Potential  also  does  not  display  information  on  the  type  or 
severity.  As  stated  in  paragraph  5. 2. 1.4,  meteorological 
symbolism  could  have  been  included  with  the  graphical  AIRMET  to 
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identify  the  type  and  severity  of  icing.  In  addition,  two 
interpretation  issues  were  identified  as  follows: 

a.  Presentation  of  the  Icing  AIRMETs  and  the  automated 
Icing  Potential  often  results  in  conflicting  information.  The 
gridded  icing  areas  do  not  always  agree  with  the  Icing  AIRMETs. 

If  both  components  are  presented,  they  must  be  reconciled  so  that 
conflicting  information  is  not  displayed. 

b.  Additional  interpretation  is  needed  since  only  the 
icing  potential  at  a  particular  flight  level  is  shown  in  the 
horizontal  view.  It  is  not  apparent  to  the  user  whether  icing 
areas  may  exist  directly  above  or  below  the  selected  flight 
level.  This  may  be  partially  rectified  by  selecting  a  route  of 
flight  and  viewing  a  vertical  cross  section.  For  example,  figure 
7  displays  a  route  of  flight  in  the  horizontal  view  from  eastern 
Maine  across  northern  New  York.  The  only  area  of  icing  potential 
is  the  gridded  shaded  area  in  northern  New  Hampshire.  However, 
the  vertical  cross  section  displays  another  area  of  icing  only 
about  1,000  feet  below  the  selected  flight  level.  The  vertical 
cross  section  enables  the  identification  of  icing  potential  areas 
above  and  below  the  flight  level.  However,  limitations  still 
exist  since  icing  potential  areas  may  exist  to  either  side  of  the 
route  and  these  will  not  be  displayed  on  the  vertical  cross 
section.  In  order  to  know  fully  what  is  occurring  along  and  in 
the  vicinity  of  the  route  of  flight,  it  is  necessary  to  scan 
through  several  flight  levels,  use  a  vertical  cross  section,  and 
then  mentally  interpolate  areas  of  iqing  identified  in  the  scans 
to  locations  around  the  route.  A  mosaic  of  icing  potential 
horizontally  and  vertically  around  the  route  of  flight  may 
provide  a  solution  to  this  time  consuming  process.  Since 
information  on  icing  type  and  severity  is  not  provided,  together 
with  conflicting  information  and  the  need  to  scan  several  layers 
in  conjunction  with  a  vertical  cross  section,  the  Inflight  Icing 
Potential  level  of  interpretation  is  assigned  a  numerical  rating 
of  8 . 

S  ■  2  ■  1 . 7  Route  Winds . 

While  stated  as  a  separate  product,  the  Route  Winds  are  more 
suitable  as  an  option  of  the  Winds  Aloft  product.  After  a  route 
of  flight  is  identified,  selecting  the  Route  Winds  option  will 
display,  in  numeric  format,  the  average  winds  along  legs  of  the 
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flight .  The  only  apparent  interpretation  issue  is  determining 
what  exact  distance  each  average  wind  applies  to  over  the  flight 
route.  No  meteorological  symbolism  is  used.  The  Route  Winds 
level  of  interpretation  is  assigned  a  numerical  rating  of  2 . 

5.2.2  Improvement  Over  Current  Products. 

In  the  following  paragraphs,  the  AWRP  is  compared  to  current 
operational  products  viewed  at  the  Millville,  New  Jersey,  AFSS. 
While  products  may  vary  between  AFSSs,  the  products  viewed  at 
Millville  are  considered  representative  of  current  operational 
capabilities.  In  each  paragraph  the  AFSS  products  are 
summarized,  followed  by  differences  and  improvements  found  in  the 
AWRP. 

5 . 2 . 2 . 1  Radar  Mosaic  and  Storm  Attributes . 

The  current  operational  AFSS  radar  product  is  a  national  radar 
mosaic  that  is  updated  every  5  minutes.  Predefined  regions  can 
be  selected  in  order  to  simulate  a  zoom  capability.  Looping  of 
successive  images  is  available  in  order  to  estimate  movement, 
growth,  and  decay  of  precipitation  areas.  Individual  radar  sites 
may  be  selected  for  view;  however,  the  selection  process  takes  3 
to  6  minutes,  which  makes  the  capability  virtually  useless.  The 
AWRP  Radar  Mosaic  is  similar  to  the  current  operational  AFSS 
radar  mosaic  with  the  exceptions  of  no  looping  capability,  and 
the  inclusion  of  lightning  information,  storm  attributes  (i.e., 
storm  motion)  and  the  AWRP  display  features  such  as  zoom  and  pan. 
The  lightning  information  allows  the  identification  of 
thunderstorm  activity  versus  only  the  existence  of  precipitation. 
The  automated  AWRP  storm  motion  information  removes  the  user 
interpretation  needed  when  using  the  current  looping  capability. 
However,  in  order  for  the  motion  information  to  be  an 
improvement,  the  inconsistencies  discussed  in  paragraph  5. 2. 1.1 
must  be  corrected.  One  of  the  significant  improvements  of  the 
AWRP  Radar  Mosaic  is  the  inclusion  of  the  zoom  function  and  the 
corresponding  rapid  response  time  (on  the  order  of  1  second)  to 
zoom  in  on  an  area.  It  should  be  noted  the  zoom  function  is  not 
unique  to  the  AWRP.  This  function  is  available  on  many  graphical 
weather  systems  currently  provided  by  commercial  vendors. 
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5. 2. 2. 2  Winds  Aloft. 


Current  operational  AFSS  Winds  Aloft  products  display  wind 
direction  and  speed  represented  by  a  staff  with  barbs  attached. 
The  products  are  limited  to  three  vertical  levels  for  analysis 
conditions  and  eight  vertical  levels  for  forecast  conditions  that 
are  only  valid  at  12  hours  from  the  twice  daily  analysis  times. 

In  AFSS  operations,  it  is  often  faster  to  use  textual  wind 
information  for  specific  locations  rather  than  interpreting 
graphical  data  to  locations  of  interest.  The  AWRP  Winds  Aloft 
offers  improved  spatial  and  temporal  resolution.  Vertical 
resolution  is  on  the  order  of  1,000  to  2,000  feet  with  25  levels 
available  both  for  the  analysis  and  forecast .  Forecasts  are 
available  on  an  hourly  basis  with  updates  computed  every  3  hours. 
Horizontal  resolution  is  improved,  thus  decreasing  the  amount  of 
interpretation  that  must  be  done  in  order  to  determine  winds 
along  a  route  that  may  fall  in  between  grid  points. 

5. 2. 2. 3  Temperatures  Aloft. 

As  with  the  Winds  Aloft,  current  operational  AFSS  Temperatures 
Aloft  products  are  presented  at  three  vertical  levels  for 
analysis  and  eight  vertical  levels  for  forecasts  that  are  valid 
only  at  12  hours  from  the  twice  daily  analysis  times.  Spatial 
and  temporal  resolutions  are  the  main  deficiencies  in  current 
Temperatures  Aloft  products .  The  AWRP  Temperatures  Aloft 
overcomes  the  resolution  deficiencies,  but  is  limited  by  the 
level  of  interpretation  needed  as  a  result  of  the  absence  of 
display  information  over  large  regions  of  the  country  (see 
paragraph  5. 2. 1.3).  Thus,  resolution  is  improved  over  existing 
products,  but  the  utility  of  the  Temperatures  Aloft  product  is 
questionable  due  to  the  high  level  of  interpretation  required. 

5 .2 .2 .4  ATRMETs  and  SIGMETs. 

Current  operational  AFSS  graphical  AIRMETs  and  SIGMETs  are 
created  by  manually  plotting  text  information  using  a  graphics 
pad.  Along  with  the  area  outlines  of  each  AIRMET  or  SIGMET  is 
plotted  text  information  identifying  the  particular  phenomena  and 
characteristics  (e.g,  severity,  tops,  and  bases).  Updates  are 
manually  created,  thus  there  exists  the  possibility  of  missing  an 
update  or  an  initial  release.  The  AWRP  AIRMETs  and  SIGMETs  are 
entirely  automated  thus  removing  the  possibility  of  missing  an 
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AIRMET  or  SIGMET.  However,  the  AWRP  display  lacks  any 
characteristic  information.  Characteristics  about  the  AIRMET  or 
SIGMET  have  to  be  found  by  the  user  scanning  through  text 
information.  This  effectively  nullifies  much  of  the  improvement 
over  current  products. 

5. 2. 2. 5  Current  Flight  Categoiry. 

Currently,  no  AFSS  flight  category  product  exists.  The  closest 
operational  product  is  the  surface  weather  depiction  which  shows 
the  position  of  high  and  low  pressure  systems,  weather  fronts, 
and  current  weather  by  using  weather  symbology,  as  well  as 
contours  of  MVFR  and  IFR.  The  surface  weather  depiction  is 
issued  every  3  hours.  For  more  recent  updates,  AFSS  personnel 
must  consult  text  information  from  locations  of  interest.  The 
AWRP  Current  Flight  Category  gives  automatic  updates  as  reports 
become  available  on  conditions  at  airport  locations.  No 
contouring  is  provided  by  the  AWRP,  although  it  is  expected  that 
mental  contouring  will  occur  over  regions  of  concern. 

5. 2. 2. 6  Inflight  Icing  Potentjial.. 

No  current  capabilities  exist  other  than  the  plotting  of  AIRMETs 
and  SIGMETs  discussed  in  paragraph  5. 2. 2. 4.  The  AWRP  presents  a 
new  capability.  However,  since  Inflight  Icing  Potential  gives  no 
information  on  the  type  and  severity  of  icing,  improvements  are 
predominantly  with  the  automation  of  the  Icing  AIRMETs. 

5 . 2  ■  2 ■ 7  Route  Winds . 

Current  operational  capabilities  for  Route  Winds  consists  of  text 
information  listing  winds  aloft  at  specific  stations  along  a 
route  of  flight.  These  stations  may  or  may  not  be  evenly 
distributed  along  the  route  and  may  be  several  hours  different 
from  the  time  of  flight.  The  AWRP  Route  Winds  are  displayed  in 
the  same  numeric  format  as  the  current  text  information. 
Improvements  over  the  current  text  product  come  from  displaying 
the  winds  in  a  graphical  manner  that  does  not  force  the  user  to 
search  through  text  material  to  find  the  relevant  information. 

In  addition,  winds  are  averaged  along  evenly  distributed  portions 
of  the  route  of  flight  and  with  hourly  forecast  data.  As  a 
result,  the  timing  of  the  winds  should  more  closely  match  the 
time  of  flight. 
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5.2.3  Meteorolocrical  Consistency. 


The  AWRP  was  evaluated  for  meteorological  consistency  between 
graphics  and  text.  Also  evaluated  was  consistency  between 
individual  products  showing  related  phenomena  over  the  same 
geographical  region.  Results  of  the  meteorological  consistency 
evaluation  are  outlined  in  table  12.  Inconsistencies  in  the  AWRP 
are  discussed  in  the  following  paragraphs. 

5 .2. 3.1  AIRMETs  and  SIQMRTs. 

a.  In  the  absence  of  radar  data,  a  user  must  rely  on 
Convective  SIGMETs  to  determine  the  location  of  thunderstorm 
activity.  The  incorrect  plotting  of  a  Convective  SIGMET  could 
have  a  significant  negative  impact  upon  aviation  operations. 
Several  occasions  were  identified  with  the  AWRP  when  Convective 
SIGMETs  were  not  plotted  correctly.  Incorrect  plotting  involved 
Convective  SIGMETs  defined  with  two  or  three  points,  along  with  a 
specified  width,  identifying  a  line  of  thunderstorms. 

1.  When  only  two  points  define  a  Convective  SIGMET, 
the  points  identify  the  endpoints  of  the  SIGMET.  The  SIGMET  is 
extended  equidistant  to  either  side  of  the  endpoints  and 
perpendicular  to  a  line  connecting  the  two  endpoints.  Situations 
were  identified  with  the  AWRP  where  the  endpoint  extensions  were 
not  perpendicular  to  the  line  connecting  the  two  endpoints.  As  a 
result,  skewed  Convective  SIGMETs  were  displayed  that  did  not 
cover  the  line  of  thunderstorms  as  was  intended. 

2 .  When  a  Convective  SIGMET  used  three  points  to 
define  a  line  of  thunderstorms,  the  AWRP  connected  the  three 
points  in  a  triangle  pattern  enclosing  an  area.  However,  the 
intention  of  a  three  point  Convective  SIGMET  defining  a  line  of 
thunderstorms  was  to  form  a  bow  shaped  line  of  the  width 
specified  in  the  SIGMET.  As  a  result  of  the  incorrect  plotting 
by  the  AWRP,  either  the  leading  edge  or  the  trailing  edge  of  a 
line  of  thunderstorms  was  not  covered  by  the  Convective  SIGMET. 

b.  According  to  the  AWRP  User's  Guide,  the  capability 
exists  to  have  text  messages  for  particular  AIRMETs  or  SIGMETs 
appear  on  the  display.  However,  when  the  process  for  retrieving 
the  text  information  is  followed,  the  text  for  multiple  AIRMETs 
and  SIGMETs  appears  on  the  display  and  the  user  has  to  search  for 
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TABLE  12.  METEOROLOGICAL  CONSISTENCY  RESULTS 


PRODUCT 

RESULT 

AIRMETS/SIGMETS 

Graphic  did  not  match  text . 

AIRMETS/SIGMETS 

Text  did  not  match  particular 

AIRMET/SIGMET  of  interest. 

AIRMETS/SIGMETS 

Text  was  not  current. 

Radar  Mosaic 

Surface  observations  within  reflectivity 
areas  did  not  indicate  precipitation. 

Radar  Mosaic 

Convective  SIGMET  did  not  correspond  to 
reflectivity. 

Radar  Mosaic 

Area  Motion  did  not  agree  with  Convective 
SIGMET. 

Winds  Aloft 

Vertical  consistency  was  observed. 

Temperatures 

Vertical  consistency  was  observed. 

Temperatures 

Agreed  with  icing  potential  temperature 
criteria. 

Flight  Category 

Graphic  did  not  match  text .  , 

Flight  Category 

Locations  were  not  plotted  correctly. 

Flight  Category 

Idle  locations  were  current. 

Route  Winds 

Agreed  with  winds  aloft . 

Cross  Section 

Did  not  agree  with  plan  view. 
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the  one  of  interest.  Since  the  AIRMETs  and  SIGMETs  are  not 
labelled  on  the  graphical  display,  the  user  has  to  resort  to 
geographical  identifiers  (usually  state  locations  or  location 
identifiers)  to  identify  the  desired  text.  Too  much  time  is 
involved  with  identifying  the  AIRMET  or  SIGMET  text  of  interest. 

c.  All  displayed  AIRMETs  and  SIGMETs  had  current  text 
information  with  the  exception  of  Convective  SIGMETs.  The  AWRP 
display  often  had  two  overlying  Convective  SIGMETs  in  a  single 
region.  When  the  text  for  the  SIGMETs  was  displayed,  it  was 
found  that  both  SIGMETs  had  valid  times,  but  one  had  been  issued 
1  hour  before  the  other.  Convective  SIGMETs  are  valid  for  2 
hours;  however,  if  after  1  hour  they  are  still  necessary,  new 
ones  are  issued  to  replace  those  issued  the  previous  hour.  All 
Convective  SIGMETs  from  the  previous  hour  are  superseded.  The 
reason  the  Convective  SIGMETs  have  a  2 -hour  valid  period  is 
strictly  due  to  the  possibility  of  communication  failures  or  AWC 
being  unable  to  issue  Convective  SIGMETs  for  a  particular  hour. 

In  these  cases,  the  Convective  SIGMET  from  the  previous  hour 
would  be  valid  for  the  full  2  hours  or  until  a  newer  one  is 
issued.  The  AWRP  should  not  display  Convective  SIGMETs  from  a 
previous  hour  when  current  ones  have  been  received. 

.5  2...R^dar . Mgg.aig. . 

a.  Level  one  and  two  reflectivity  levels  from  the  AWRP 
Radar  Mosaic  had  station  observations  within  the  reflectivity 
contours  that  did  not  report  precipitation,  but  did  report  clouds 
aloft.  Apparently,  due  to  the  distance  from  the  radar,  the 
precipitation  was  observed  aloft  by  the  radar  and  was  evaporating 
before  reaching  the  ground.  Other  isolated  cases  of  anomalous 
propagation  were  observed  where  reflectivity  was  observed 
directly  over  a  location,  but  the  station  observation  did  not 
contain  any  indication  of  precipitation  or  clouds  aloft .  While  a 
small  number  of  the  above  cases  were  observed,  for  the  most  part, 
the  Radar  Mosaic  agreed  quite  well  with  the  surface  observations . 
It  is  likely  the  anomalies  are  in  part  due  to  the  limitations  of 
the  radar  rather  than  that  of  the  AWRP. 

b.  The  issue  of  Convective  SIGMETs  not  agreeing  with  the 
radar  reflectivity  fields  is  addressed  in  paragraph  5. 2. 3.1. 
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c.  The  conflicts  identified  between  the  Radar  Mosaic  Area 
Motion  and  the  area  motion  given  in  the  Convective  SIGMET  are 
addressed  in  paragraph  5. 2. 1.1. 

5 .2. 3. 3  Current  Flight  Category . 

a.  Occasions  were  identified  when  the  wrong  color  code  was 
applied  to  individual  stations.  In  one  example,  a  station  was 
color  coded  as  VFR  (i.e.,  ceiling  greater  than  3,000  feet),  but 
the  observation  recorded  a  ceiling  of  2,700  feet,  which  would 
place  the  station  in  the  MVFR  category.  Another  example  of  the 
graphics  not  matching  the  text  was  the  display  of  the 
thunderstorm  symbol  for  a  station  whose  latest  observation 
included  only  rainshowers .  Thunderstorms  had  occurred  the 
previous  hour  at  the  location  but  had  finished  by  the  time  of  the 
observation. 

b.  Problems  with  the  location  of  stations  were  noted 
during  the  evaluation.  Many  stations  located  over  water  were 
erroneously  assigned  text  messages  for  inland  stations.  It  is 
not  clear  whether  the  problem  was  the  wrong  text  given  for  an 
actual  over  water  station  or  whether  the  station  is  incorrectly 
plotted.  The  inland  stations  that  corresponded  to  the  text  were 
found  in  their  correct  geographical  location.  Recurring  examples 
were  PHL  (Philadelphia,  Pennsylvania) ,  DOV  (Dover,  Delaware) ,  and 
SBY  (Salisbury,  Maryland)  which,  while  being  in  their  correct 
geographical  locations,  were  also  plotted  off  the  east  coast  in 
the  Atlantic  Ocean. 

c.  Stations  depicted  as  being  idle  (i.e.,  no  reports 
within  the  last  80  minutes)  often  had  current  observations 
available.  Erroneously  displaying  the  flight  category  as 
outdated  could  prompt  a  user  to  ignore  the  observation  for  a 
particular  location. 

5. 2. 3. 4  Vertical  Cross  Section. 

When  displaying  the  Inflight  Icing  Potential  product,  it  was 
noted  on  several  occasions  that  the  vertical  cross  section  did 
not  agree  with  the  horizontal  plan  view.  The  discrepancies  were 
not  noted  with  any  other  products.  Examples  of  discrepancies 
included  a  route  selected  through  a  gridded  icing  area  which  on 
the  vertical  cross  section  either  showed  the  route  under  or  over 
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(but  not  touching)  the  icing  potential  area.  This  would  indicate 
that  either  the  icing  potential  area  was  missing  from  the 
vertical  cross  section  or  the  icing  potential  area  did  not  extend 
as  far  as  the  horizontal  plan  view  indicated.  In  the  latter 
example,  a  route  was  chosen  that  began  and  ended  within  a 
continuous  icing  potential  area.  The  corresponding  vertical 
cross  section  showed  the  beginning  and  ending  of  the  route 
outside  of  icing  areas . 

5.3  SYSTEM  ADMINISTRATOR  RESULTS  AND  DISCUSSION. 

The  AWRP  Installation  Documentation  and  AWRP  System  Management 
Documentation  were  evaluated  to  ensure  accuracy  and  ease  of 
understanding  of  procedures  and  information. 

5.3.1  General  Comments. 

Sections  of  the  documentation  contained  incorrect,  inconsistent, 
and/or  missing  pieces  of  information.  Paragraph  5.3.2  will 
address  specific  comments  regarding  each  of  the  documents. 

5.3.2  Specific  Comments. 

5. 3. 2.1  AWRP  Installation  -  Executable  Only  Document. 

a.  Page  2,  section  2.5:  The  documentation  says  that  the 
live  data  AWRP  system  requires  additional  software  to  retrieve 
live  data,  but  it  does  not  say  what  software  is  needed.  Clarify 
if  this  software  is  commercial-off- the -shelf (COTS) equipment  or  it 
has  been  developed  in-house  to  suit  individual  sites. 

b.  Page  4,  section  3.0:  Step  8  consists  of  a  list  of 
directories  that  need  to  be  created.  When  executing,  the 
software  errors  are  received  indicating  that  a 

/home/awpg/live/data/ltg/wsi  directory  was  also  needed.  Add  this 
to  the  directory  list. 

5. 3. 2. 2  AWRP  Installation  -  Source  Code  Document. 

a.  Page  3,  section  3.0:  In  step  3,  the  login  name 
identified  for  the  ftp  is  incorrect. 
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b.  Page  5,  section  3.0:  kav_rm_demo . sh  and 
wsi_ltg_demo . sh  are  found  in  directories  other  than 
~/startup/demo.  A  line  needs  to  be  added  informing  the  operator 
to  change  directories  before  editing  these  files. 

c.  Page  7,  section  4.0:  Step  1  indicates  that  <user- 
path>/awpg_user/bin  should  be  added  to  the  path.  That  directory- 
structure  does  not  exist  using  the  awpg  system.  It  seems  likely 
that  awpg_user  should  be  replaced  by  awpg_version. 

d.  Page  9,  section  7.1:  The  document  states  that  some 
executables  are  not  accompanied  by  their  source  code  as  part  Of 
the  AWRP  distribution.  If  it  is  because  this  information  is 
proprietary,  that  should  be  stated. 

5 . 3 . 2 . 3  AWRP  Live  System  Management . 

a.  Page  8,  section  4.3:  If  the  software  requires  that 
"tmp"  subdirectories  exist  under  the  data  directories  then  that 
should  be  specified.  If  the  "tmp"  directories  contain  data,  at 
some  point  that  should  also  be  discussed. 

b.  Page  14,  section  6.2:  The  following  should  be  briefly 
defined  one  at  a  time:  Type,  subtype,  instance,  host,  port, 
data.  The  information  currently  listed  should  be  kept,  but  that 
information  needs  to  be  expanded. 

c.  Page  15,  section  6.3:  Each  heading  (type,  subtype, 
name,  packet id,  last  recv)  should  be  defined. 

6.  CONCLUSIONS. 

6.1  HUMAN  FACTORS  CONCLUSIONS . 

The  Aviation  Weather  Research  Products (AWRP) ,  when  considered 
individually  and  collectively  as  a  single  system,  would  provide 
improvements  over  current  systems .  However,  if  products  are  to 
be  considered  for  potential  incorporation  into  FAA  systems, 
considerable  additions  and  improvements  would  be  necessary  to 
address  user  needs  and  requests.  The  following  paragraphs 
outline  improvements  for  future  development. 
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.  l  _.General  Review  Stage. 


The  AWRP  interface  and  User's  Guide  are  summarized  in  the 
following  paragraphs. 

a.  The  AWRP  display  and  interface  follow  many  human 
factors  standards  and  guidelines;  however,  the  display  does  need 
improvement  in  several  areas  as  noted  in  paragraph  5. 1.1.1.  The 
interface  should  reflect  job  tasks  and  follow  good  design 
principles .  A  usable  interface  can  improve  performance  and 
decrease  controller  error  during  critical  high  traffic  periods. 

b.  The  AWRP  User's  Guide  provided  a  good  description  of 
the  products;  however,  instructions  regarding  the  user  interface 
tended  to  be  inconsistent  and  unclear.  User  guides  are  utilized 
as  references  and  are  rarely  read  in  their  entirety.  Therefore, 
the  guides  should  provide  clear  and  concise  explanations. 
Discussions  of  products  and  features  should  provide  all  required 
information.  Referencing  different  sections  should  be  kept  to  a 
minimum . 

6.1.2  Operational  Analysis  Stage. 

The  operational  analysis  stage  was  conducted  in  order  to 
determine  how  well  the  AWRP  would  perform  in  an  operational 
environment . 

6..  1  .■2.._1  Air  .Traffic  User  Working  Group  (ATUWG)  and  1993  Demval 

Requests • 

Users  requested  several  products  and  features  during  ATUWG 
meetings  and  the  1993  Demval.  The  AWRP  incorporated  about  half 
of  the  user  requests  from  the  ATUWG  meetings.  Color  coding  for 
each  of  the  products  and  features  were  recommended  by  users . 
Several  of  these  recommendations  were  not  implemented. 
Additionally,  several  requests  from  the  1993  Demval  were  not 
incorporated  in  the  AWRP. 

While  some  user  requests  may  be  beyond  the  scope  of  the  AWRP, 
they  are  based  upon  the  needs  of  specialists  and  controllers  in 
an  operational  setting.  User  meetings  were  intended  to  provide 
developers  with  feedback  and  weather  information  needs. 

Attention  must  be  given  to  this  information  in  the  development  of 
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future  products  in  order  to  provide  specialists  with  the  required 
data  presented  in  a  useful  format.  Future  systems  intended  for 
operational  use  will  require  these  features. 

fi.l.2.2  Job  Task  Analysis  (JTA)  Weather  Information  Requirements. 

The  JTA  comparison  identified  the  following  items: 

a.  Many  pieces  of  weather  information  required  by 
Automated  Flight  Service  Station  (AFSS)  Preflight,  Inflight,  and 
Enroute  Flight  Advisory  Specialists,  as  well  as  Air  Route  Traffic 
Control  Center  (ARTCC)  Traffic  Management  Coordinators  (TMC)  were 
not  available  on  the  AWRP.  This  information  is  required  to 
satisfy  the  needs  of  AFSS  and  ARTCC  Traffic  Management  Unit  (TMU) 
operational  tasks. 

b.  The  AWRP  was  assessed  according  to  how  well  it  overcame 
problems  on  current  weather  systems  regarding  information 
presentation  and  retrieval.  The  AWRP  automatically  plots  weather 
advisories  and  provides  winds  aloft  for  more  flight  levels  than 
current  capabilities.  However,  the  technology  to  parse 
advisories  and  plot  information  is  not  unique  to  the  AWRP. 
Additionally,  the  ability  to  obtain  winds  aloft  at  more  flight 
levels  is  a  function  of  improved  weather  prediction  models. 

c.  The  AWRP  does  not  overcome  the  JTA  identified  problems 
of  comparing  textual  and  graphical  information,  needing  a 
graphical  presentation  of  Pilot  Reports  (PIREPs) ,  dealing  with 
extensive  textual  information,  or  dealing  with  different 
interface  formats.  In  order  to  acquire  all  required  weather 
information,  users  would  have  to  access  other  weather  information 
systems.  Additionally,  the  AWRP  is  not  tailored  to  meet  users' 
job  tasks.  In  developing  products  to  meet  the  needs  for  two  user 
groups,  the  AWRP  does  not  fully  meet  the  needs  of  either  user 
group. 

While  it  is  recognized  that  the  current  AWRP  was  not  intended  for 
operational  use,  operational  requirements  and  users'  needs  were 
identified  throughout  the  development  of  the  AWRP.  Failure  to 
completely  follow  operational  requirements  and  users '  needs  will 
result  in  future  systems  that  do  not  provide  the  required  weather 
information  or  present  that  information  according  to  users' 
tasks . 
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The  meteorological  evaluation  identified  improved  data  resolution 
and  timeliness,  and  application  of  AWRP  functions  (e.g.,  zoom)  as 
positive  attributes  of  the  AWRP.  The  improved  data  resolution 
and  timeliness  are  a  result  of  improved  weather  prediction 
models.  The  positive  functions  are  available  on  many  commercial 
vendor  graphical  weather  systems.  In  addition,  the  real-time 
storm  motions  and  forecasted  icing  potential  of  the  AWRP  offer 
capabilities  not  currently  available  to  operational  users. 

However,  the  meteorological  evaluation  identified  significant 
interpretation  issues  concerning  conflicting,  absent,  and 
fluctuating  information.  Specific  conclusions  about  each  phase 
of  the  meteorological  evaluation  are  stated  in  the  following 
paragraphs . 

&^2..1  Meteorological  Interpretation. 

a.  Conflicting  information  was  presented  when  a  mixture  of 
automated  and  human  derived  products  existed.  For  operational 
use,  the  two  types  of  products  must  be  reconciled.  Otherwise, 
significant  differences  for  the  same  parameter  can  be  displayed 
leading  to  user  confusion. 

b.  Graphical  information  was  lacking.  The  graphical 
display  of  Airmen  Meteorological  Statements  (AIRMETs)  and 
Significant  Meteorological  Statements  (SIGMETs)  did  not  include 
characteristics  of  the  phenomena  of  interest,  such  as  vertical 
extent,  type,  and  severity.  Current  operational  graphical  AFSS 
products,  include  these  characteristics. 

c.  The  Flight  Category  product  lacks  forecast  information. 
With  only  current  observations,  other  weather  information  sources 
must  be  used  in  order  to  obtain  the  required  forecast 
information. 

d.  The  widely  fluctuating  storm  area  motions  will  likely 
lead  to  confusion.  In  addition,  wide  fluctuations  tend  to  reduce 
the  credibility  of  a  product. 

e.  The  AWRP  display  lacks  information  over  large  regions. 
Temperature  contours  have  such  wide  spacing  that  significant 
interpretation  was  required  in  order  to  deteinnine  temperature  at 
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locations.  Flight  Category  information  was  only  available  for 
locations  with  hourly  surface  weather  observations.  No 
information  was  available  at  other  locations. 

f.  The  limitation  of  the  horizontal  plan  view  to  a 
particular  flight  level  limits  the  available  information. 
Potential  hazardous  weather  occurring  directly  below  or  above  a 
flight  level  is  not  identified  unless  a  user  scans  through 
multiple  flight  levels.  The  user  must  mentally  interpolate  and 
combine  multiple  levels  into  a  mental  composite  picture. 

6.2.2  Improvements. 

Overall,  it  must  be  concluded  that  the  AWRP  offers  few 
improvements  over  existing  AFSS  operational  capabilities.  The 
majority  of  the  AWRP  improvements  were  due  to  software  functions 
that  are  available  through  commercial  vendors  and  improved 
weather  prediction  models  that  provide  input  to  the  AWRP.  The 
weather  prediction  models  will  likely  provide  more  timely  updates 
to  existing  operational  platforms  in  the  near  future. 

a .  The  AWRP  demonstrated  improvement  over  current  AFSS 
capabilities  by  automating  tasks  that  are  currently  performed  by 
manual  methods.  The  benefits  are  less  interpretation,  real-time 
availability,  and  avoidance  of  possible  omissions  of  information 
to  Users. 

b.  The  AWRP  functions  offer  improvement  over  existing  AFSS 
capabilities.  However,  commercial  vendor  graphical  weather 
systems  have  similar  functions. 

c.  The  AWRP  has  improvements  in  spatial  and  temporal  data 
resolution  which  are  the  result  of  improved  weather  prediction 
models  that  provide  input  to  the  AWRP.  It  is  expected  that 
commercial  vendors  will  also  take  advantage  of  these  resolution 
improvements. 

d.  The  AWRP  does  not  provide  descriptive  information  on 
the  graphical  display  of  AIRMETs  and  SIGMETs  which  forces  the 
user  to  search  through  textual  information. 
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6.2.3  Meteorological  Consistency. 


During  the  AWRP  evaluation,  some  inconsistencies  were  identified. 
If  the  AWRP  were  an  operational  system,  the  inconsistencies  would 
be  sufficient  to  cast  doubt  upon  the  reliability  and  accuracy  of 
the  AWRP.  The  major  inconsistency  was  between  the  automated  and 
human  derived  products  as  discussed  previously  in  paragraph 
6. 2.1. a.  The  remainder  of  the  discrepancies  identified  in  the 
evaluation  appeared  to  be  caused  by  software  errors  that  may  be 
correctable  by  minor  fixes.  Examples  of  inconsistencies  were 
discussed  in  paragraph  5.2.3. 

■6.3  SYSTEM  ADMINISTRATOR  CONCLUSIONS. 

Inconsistencies  in  the  AWRP  installation  and  system  management 
documentation  were  found.  Such  inconsistencies  included: 

a.  Missing  directory  paths, 

b.  Incorrect  directory  listings,  and 

c.  Lack  of  header  definitions. 

Examples  of  inconsistencies  were  discussed  in  paragraph  5.3.2. 

7.  RECOMMENDATIONS. 

a.  The  display  and  interface  should  be  improved  if  there 
is  any  consideration  for  use  as  a  stand-alone  system.  However, 
if  products  are  going  to  be  ported  to  a  different  platform  (i.e., 
vendor  systems) ,  improving  the  current  interface  would  not  be 
necessary.  Problems  with  the  current  display  and  interface 
should  be  provided  to  CRDA  vendors  to  ensure  these  problems  are 
resolved  in  any  future  implementation  of  the  products. 

b.  Improvements  should  be  made  to  the  User's  Guide 
regarding  manipulation  of  the  interface. 

c.  User  requests  and  feedback  should  be  utilized  in 
product  development.  The  purpose  of  these  meetings  was  to 
provide  feedback  regarding  display  format  and  weather  information 
needs  of  users.  This  information  should  be  used  as  the  primary 
guidance  in  the  development  of  the  AWRP  display. 
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d.  Graphics  need  to  present  as  much  information  as 
possible  without  forcing  the  user  to  search  through  text  to  find 
significant  information. 

e.  As  noted  in  the  1993  Demval  Report,  products  and 
functions  should  be  tailored  for  users. 

f.  Conflicting  meteorological  information  must  be  avoided. 
Automated  and  human  derived  products  must  be  reconciled  if  both 
are  going  to  be  displayed  to  nonmeteorologists.  Either  the 
automated  should  be  used  as  guidance  for  a  meteorologist  in 
developing  a  product  sent  to  the  end  user  or  the  human  derived 
product  should  be  replaced  with  the  automated  product. 

g.  Automated  meteorological  forecasting  techniques  must 
produce  stable  results  and  minimize  wide  fluctuations  that  are 
difficult  to  interpret. 

h.  Thorough  independent  testing  of  the  AWRP  should  be 
conducted  before  any  new  release  of  the  software  to  CRDA  vendors . 
The  version  of  the  AWRP  evaluated  in  this  report  was  released  to 
CRDA  vendors  for  possible  integration  onto  existing  platforms. 
Products  having  the  amount  of  inconsistencies  identified  in  the 
AWRP  should  not  be  considered  for  operational  use. 

i.  Improvements  should  be  made  to  the  AWRP  Installation 
and  System  Management  Documentation. 

Despite  the  discrepancies  noted,  the  AWRP  products  have  the 
potential  to  provide  controllers  with  much  needed  improvements  in 
the  area  of  Aviation  Weather  Information.  Continued  development 
of  aviation  weather  graphical  products  and  the  underlying  science 
is  strongly  recommended.  However,  consideration  should  be  given 
to  off-the-shelf  commercial  (COTS)  weather  products  or  the 
development  of  weather  graphics  by  commercial  vendors.  NCAR's 
involvement  in  the  rendering  of  products  rather  than  the 
development  of  aviation  weather  science  may  not  be  the  most 
efficient  use  of  resources. 
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AFOTEC 

- 

Air  Force  Operational  Test  and  Evaluation  Center 

AFSS 

- 

Automated  Flight  Service  Station 

AGFS 

- 

Aviation  Gridded  Forecast  System 

AIRMET 

- 

Airmen  Meteorological  Information 

ARTCC 

- 

Air  Route  Traffic  Control  Center 

ASD 

- 

Aircraft  Situation  Display 

ASP 

- 

Airport  Sequencing  Program 

ATUWG 

- 

Air  Traffic  User  Working  Group 

AWPG 

- 

Aviation  Weather  Products  Generator 

AWC 

- 

Aviation  Weather  Center 

AWR 

- 

Aviation  Weather  Research 

AWRP 

- 

Aviation  Weather  Research  Products 

COTS 

- 

Commercial-off- the -shelf 

CRDA 

- 

Cooperative  Research  and  Development  Agreement 

CSU 

- 

Channel  Service  Unit 

CWA 

- 

Center  Weather  Advisory 

CWSU 

- 

Center  Weather  Service  Unit 

DEMVAL 

- 

Demonstration  and  Evaluation 

DME 

- 

Distance  Measuring  Equipment 

DOV 

- 

Dover,  Delaware 

DSTN 

- 

Destination 

DSU 

- 

Data  Service  Unit 

EFAS 

- 

Enroute  Flight  Advisory  Specialist 

ETA 

- 

Estimated  Time  of  Arrival 

ETD 

- 

Estimated  Time  of  Departure 

FAA 

- 

Federal  Aviation  Administration 

FOS 

- 

Family  Of  Services 

FSL 

- 

Forecast  Systems  Laboratory 

FT 

- 

Terminal  Forecast 

GA 

- 

General  Aviation 

GOES 

- 

Geostationary  Operational  Environmental  Satellite 

GUI 

- 

Graphical  User  Interface 

ID 

- 

Identification 

IFR 

- 

Instrument  Flight  Rules 

ILS 

- 

Instrument  Landing  System 

IPT 

- 

Integrated  Product  Team 

ITWS 

- 

Integrated  Terminal  Weather  System 

JTA 

- 

Job  Task  Analysis 

K  Index 

- 

Lifted  Index 

KBPS 

- 

Kilobytes  per  second 
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KBVT 

km 

LAA 

LIFR 

LLWAS 

LOCN 

MIFC 

MKE 

MVFR 

MWP 

NAVAID 

NCAR 

NEXRAD 

NOTAM 

NWS 

PC 

PHL 

PIREP 

RABAT 

RARER 

RH 

RVR 

RW 

SAG 

SBY  . 

SIGMET 

TACAN 

TDWR 

TMC 

TMU 

TRACON 

TWEB 

VFR 

VHF 

VOR 

VORTAC 

WA 

WARP 

WS 

WX 


Keyboard  Video  Terminal 
Kilometers 

Local  Airport  Advisory 

Low  Instrument  Flight  Rules 

Low  Level  Windshear  Alert  System 

Location 

Model  1  Full  Capacity 
Milwaukee,  Wisconsin 
Marginal  Visual  Flight  Rules 
Meteorologist  Weather  Processor 
Navigational  Aid 

National  Center  for  Atmospheric  Research 

Next  Generation  Radar 

Notice  to  Airmen 

National  Weather  Service 

Personal  Computer 

Philadelphia,  Pennsylvania 

Pilot  Report 

Radio  Sound  Observation  Data 
Radar  Report 
Relative  Humidity 
Runway  Visual  Range 
Runway  Visibility  Value 
Surface  Area  Observation 
Salisbury,  Maryland 

Significant  Meteorological  Information 
Tactical  Air  Navigation  Station 
Terminal  Doppler  Weather  Radar 
Traffic  Management  Coordinator 
Traffic  Management  Unit 
Terminal  Radar  Approach  Control 
Transcribed  Weather  Broadcast 
Visual  Flight  Rules 
Very  High  Frequency 
VHF  Omnidirectional  Range 
VHF  Omnidirectional  Range  TACAN 
Airmet 

Weather  And  Radar  Processor 

Sigmet 

Weather 
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